A virtual space interaction method, editing and updating method, system, electronic device and readable storage medium based on AR technology

Abstract

The application relates to a virtual space interaction method, an editing and updating method, a system, electronic equipment and a readable storage medium based on AR technology, the virtual space interaction method comprising the following steps: acquiring view area information; acquiring positioning information, determining a working reference surface based on the corresponding relationship between the positioning information and a service area; determining a display direction of the working reference surface based on the positioning information to determine a display surface; acquiring real-time live image information and matching the working reference surface with the real-time live image information; loading materials in an editing operation area and adjusting the materials and playing time sequences based on preset requirements; superimposing output image information of display surface content of a visual operation area on the real-time live image information; and displaying the superimposed image on a screen of terminal equipment. The application has the advantage that an augmented reality picture can be synchronously displayed on different terminals.

Description

Technical Field

[0001] The present application relates to the field of augmented reality, and in particular to a virtual space interaction method, editing and updating method, system, electronic device and readable storage medium based on AR technology. Background Art

[0002] For a long time, landmark buildings or regional business cards in various places have usually been used as tourist check-in points, such as the Forbidden City in Beijing, the Bund in Shanghai, and the Little Mermaid in Guangzhou. During certain periods of time, they will launch light shows, fireworks shows, drones and other activities to enhance the tourist experience and boost the local tourism industry.

[0003] Augmented Reality (AR) is a new technology that seamlessly integrates real-world and virtual-world information. Using computers and other scientific technologies, it simulates and superimposes physical information (visual information, sound, smell, touch, etc., primarily visual) that is difficult to experience within a specific time and space in the real world. This virtual information is then applied to the real world and perceived by human senses, achieving a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same screen or space in real time, coexisting simultaneously. It not only displays real-world information but also virtual information, with the two types of information complementing and overlapping each other.

[0004] In related technologies, augmented reality technology can be used to overlay real and virtual scenes on mobile phones, AR glasses, and other terminal devices. However, this technology is limited to small scenes and is difficult to mass-produce and replicate. Currently, there are almost no city-level AR applications. The inventor believes that the difficulty lies in the special nature of urban and tourist attraction services, which require high synchronization and immediacy. The scenes viewed by tourists must be unified in real time. However, current augmented reality technology cannot present different content for the same attraction in the same city, or the same content for different attractions in different cities.

[0005] In the first related technology, augmented reality technology is combined with image recognition technology. This method usually requires the device to first model the acquired image and then combine it with the 3D model. It has the characteristic of good model fit, but the current real-time modeling technology is still immature, and it requires high computing power of the device and generates high latency. It is also difficult to achieve image synchronization on various devices.

[0006] In related technology 2, augmented reality technology is used in conjunction with a visual system. Images are captured by the device and the edited 2D model is displayed as a texture on the AR display device, integrating it with the real scene. However, this method is only suitable for small scenes within a single device. It is difficult to synchronize multiple images or multiple cities in large real-world scenes. Summary of the Invention

[0007] In order to enable augmented reality images to be displayed synchronously on different device terminals, the present application provides a virtual space interaction method, editing and updating method, system, electronic device and readable storage medium based on AR technology.

[0008] In the first aspect, the present application provides a virtual space interaction method based on AR technology, which adopts the following technical solutions:

[0009] A virtual space interaction method based on AR technology includes the following steps:

[0010] Obtain viewing area information, where the viewing area is divided into several service areas. Each service area corresponds to a base map reference plane containing location information. Points on the base map reference plane correspond to editing operation areas for placing materials.

[0011] Acquire positioning information, and determine a working reference plane based on a correspondence between the positioning information and the service area, wherein the working reference plane is a base map reference plane corresponding to the positioning information;

[0012] Determining the display orientation of the working reference surface based on the positioning information to determine the display surface;

[0013] Acquire real-time on-site image information and match the working reference surface with the real-time on-site image information;

[0014] Load the material into the editing area and adjust the material and playback timing based on preset requirements;

[0015] Overlaying the display surface content of the visual operation area on the real-time on-site image information as output image information, wherein the visual operation area is the editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image;

[0016] The superimposed image is displayed on the screen of the terminal device.

[0017] By adopting the above technical solution, the viewing area is usually an area suitable for tourists to view the scenery. Since the viewing area has a certain area and has a certain extension in a certain direction, the viewing effect of the target at different positions is different. If the same material is placed, dislocation and unreality will occur. The viewing area is divided into multiple service areas. Since the service areas are relatively small, based on the principle of perspective, when viewing a large target from a distance, a small distance movement has little effect on the visual effect. Therefore, the visual effect of viewing the superimposed picture of the target and the virtual image in a single service area is close and relatively real. Since there is a certain interval between the service areas, the distance generated is enough to have a large impact on the visual effect, such as visual offset, so different service areas should have corresponding different base map datum surfaces.

[0018] The device determines the service area through the positioning information, and determines the material information and the corresponding preset playing time sequence of the editing operation area to be loaded on the working reference plane of the device based on the service area. Different users using different terminals have the same time sequence of the viewed material even if they are in different service areas. In order to ensure the uniformity of the viewing effect, on the one hand, the working reference plane needs to be fine-tuned to the display orientation to adapt to the difference in camera angle at different positions in the same service area, and on the other hand, the working reference plane needs to be coordinated with the real-time image to adapt to the lens posture and lens parameters of different terminals.

[0019] Finally, since the lens range is difficult to cover the entire base map reference plane, the editing operation area corresponding to the lens range is a visible operation area, and only the material playing effect in the visible operation area needs to be mapped on the real-time image. The application program interface of the camera software or the system-level application program interface of the display screen is called to display the superimposed image on the display screen.

[0020] In summary, the scheme unifies the playing time sequence of different devices, fine-tunes the material playing effect in a small range, and adjusts the resource playing effect in a large range based on the positioning information, so as to unify the space and time of the enhanced picture of the tourism scene.

[0021] Optionally, the editing operation area includes a two-dimensional plane with a direction-adjusting reference point, and the direction-adjusting reference point is located on the base map reference plane.

[0022] By adopting the above technical scheme, the two-dimensional material can be placed in the editing operation area for editing operations such as stretching, scaling, rotating, and / or translating. The editing operation area can be coincident with the base map reference plane or at an angle with the base map reference plane. When in use, the editing operation area can be rotated around the direction-adjusting reference point to adjust the orientation of the two-dimensional material. The direction-adjusting reference point is also used to position the editing operation area on the base map reference plane, so that the editing operation area can move synchronously when the base map reference plane moves or rotates.

[0023] Optionally, the editing operation area includes a three-dimensional space with an anchoring reference point, and the anchoring reference point is located on the base map reference plane.

[0024] By adopting the above technical scheme, the three-dimensional material can be placed in the editing operation area for editing operations such as stretching, scaling, rotating, and / or translating. The anchoring reference point is used to position the editing operation area on the base map reference plane, so that the editing operation area can move synchronously when the base map reference plane moves or rotates.

[0025] Optionally, the step of acquiring the positioning information and determining the working reference plane based on the corresponding relationship between the positioning information and the service area includes:

[0026] Obtain real-time satellite positioning coordinate information and determine the service area based on the correlation between the distance difference between the satellite positioning coordinate point and the center coordinate point of each service area and the preset radius; where the center coordinate of the service area is the coordinate of the center of the service area, and the preset radius is the radius length of the service area;

[0027] Obtain the basemap datum corresponding to the service area and load it as the working datum.

[0028] Optionally, the step of obtaining positioning information and determining the working reference plane based on the correspondence between the positioning information and the service area also includes: controlling the opening and closing of the camera module based on the relative position relationship between the satellite positioning coordinate point and the service area.

[0029] By adopting the above technical solution, when the user leaves the service area, the closing action of the camera module can produce a prompt effect for the user. At the same time, the opening and closing of the camera module can also produce a guiding effect for the user to find and enter the service area.

[0030] Optionally, the step of determining the display orientation of the working reference surface based on the positioning information includes:

[0031] The display orientation of the working reference plane is adjusted based on the relative relationship between the reference orientation information and the real-time orientation information; wherein, the reference orientation information is the orientation information of the service area center coordinate point relative to the virtual coordinate point, the real-time orientation information is the orientation information of the terminal position coordinate point relative to the virtual coordinate point, and the virtual coordinate point is the reference point for establishing the base map reference plane.

[0032] By adopting the above technical solution, since the service area has a certain area, the playback image of the distant material viewed at different locations within the same service area will be slightly different. This difference is smaller the closer to the center of the base map reference plane, and larger the further away from the center of the base map reference plane. To compensate for this visual difference, the display orientation can be adaptively adjusted based on the relative relationship between the reference orientation information and the real-time orientation information.

[0033] Optionally, the step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information includes:

[0034] Acquire information of simulated positioning points within the service area, wherein each service area has multiple simulated positioning points;

[0035] Obtain the simulated positioning point closest to the satellite positioning coordinate point and use it as the terminal position coordinate point;

[0036] The preset orientation information corresponding to the terminal position coordinate point is read to determine the display orientation of the working reference surface, wherein the preset orientation information is pre-set information of the orientation of the base map reference surface with respect to the virtual coordinate point.

[0037] By adopting this technical solution, satellite positioning is highly sensitive and prone to small-scale movement when encountering interference. When the user's position drifts, it's actually moving around the simulated positioning point. Therefore, the simulated positioning point acts as an anchor, and the small area around the simulated positioning point adopts the preset orientation corresponding to the simulated positioning point. Because the display orientation corresponding to the preset orientation is pre-set, there's no need for backend recalculation or adjustment of the working reference surface, resulting in reduced computing power and faster response times.

[0038] Optionally, the step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information includes:

[0039] Obtain real-time gyroscope information and determine whether the device enters or exits the raised state based on the real-time gyroscope information;

[0040] Obtain the satellite positioning coordinate point corresponding to the moment the device enters the raised state and lock it as the terminal position coordinate point;

[0041] Calculate the angle between the real-time orientation and the reference orientation, and adjust the displayed orientation of the working reference plane based on the angle.

[0042] Optionally, the step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information further includes:

[0043] Unlock the terminal position coordinate point when the device exits the raised state.

[0044] To ensure the best possible photo or viewing experience, users typically stop moving or make small adjustments when raising their phone to shoot. Therefore, by adopting the above technical solution, when the device is detected to be raised, the satellite positioning coordinates are locked to prevent the material's position from jumping due to the adaptive adjustment of the display orientation of the working reference surface during small movements. When the user is ready to adjust the position, the corresponding user action is typically to lower the device. Therefore, when the device exits the raised state, the terminal's position coordinates are locked to facilitate the acquisition of real-time satellite positioning coordinates corresponding to the next raising action.

[0045] Optionally, the step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information further includes:

[0046] Calculate the distance difference between the real-time satellite positioning coordinates and the terminal position coordinates as the real-time drift distance;

[0047] update the terminal position coordinate point based on a relative size of the real-time drift distance and the preset drift error distance, wherein the updated value of the terminal position coordinate point is a satellite positioning coordinate corresponding to the real-time drift distance exceeding the preset drift error distance.

[0048] By adopting the above technical solution, the satellite positioning has high sensitivity, and is easy to move in a small range when encountering interference. By setting the drift range, the terminal position coordinate point in the error range can be continuously locked at the same point, greatly reducing the influence of positioning drift on the selection of the display orientation. On the other hand, in fact, the change in the panoramic visual effect caused by the slight displacement of the device is also very subtle, so this scheme can greatly improve the user experience.

[0049] Optionally, the step of loading the material into the editing operation area and adjusting the material and the playing time sequence based on the preset requirement comprises:

[0050] obtaining material placement information corresponding to the editing operation area on the working reference plane, and loading the material into the editing operation area based on the material placement information, wherein the material placement information is used to achieve the preset requirement, and comprises material file, material placement orientation and / or material proportion information;

[0051] adjusting the placement angle, color and / or size of the material in the editing operation area based on the material placement information;

[0052] setting the playing time sequence of the material based on the time sequence file.

[0053] By adopting the above technical solution, when the material is loaded into the corresponding editing operation area, the material is stretched, scaled, rotated and / or translated based on the material placement information.

[0054] Optionally, the step of obtaining the real-time live image information and matching the working reference plane with the real-time live image information comprises:

[0055] obtaining real-time live image information and real-time gyroscope information, wherein the real-time live image information and the real-time gyroscope information correspond to each other;

[0056] obtaining preset gyroscope information corresponding to the working reference plane, and rotating the working reference plane in real time based on a relative relationship between the preset gyroscope information and the real-time gyroscope information;

[0057] controlling the scaling of the material in the editing operation area on the working reference plane based on the camera module parameters.

[0058] By adopting the above technical solution, the base map reference plane is set based on the base map at the beginning of the setting, and the preset gyroscope information is recorded corresponding to it, that is, the posture of the base map reference plane should correspond to the gyroscope information. During shooting, the pitch angle and left and right orientation of the device are different, so it is necessary to determine the device posture when shooting the real-time image based on the real-time gyroscope information, and use the relative relationship between the preset gyroscope information and the real-time gyroscope information to adjust the working reference plane to a posture that matches the real-time picture. In addition, since the camera module parameters of different devices are different, such as wide angle, etc., the material is scaled based on the camera module parameters so that the image content obtained by different camera modules can correspond to the material position, ensuring that the material playback screen and the real-time image content on different devices can be relatively positioned.

[0059] Optionally, the service areas are arranged at intervals and a blank buffer zone is provided between adjacent service areas.

[0060] With this technical solution, due to the high sensitivity of satellite positioning, even minor external interference can cause real-time satellite positioning coordinates to drift. When two service areas are connected and the device is located near their junction, it is easy for the device to repeatedly load the basemap corresponding to both service areas as its working reference, causing the playback image to jump repeatedly. The blank buffer can avoid this situation and also prevent the problem of excessive service areas leading to high maintenance costs.

[0061] In a second aspect, the present application provides an editing and updating method, which adopts the following technical solution:

[0062] An editing and updating method, applied to a server, comprises the following steps:

[0063] Set the viewing area to the ground Figure 2 dimensional space, and divide the viewing area into service areas and blank buffer areas;

[0064] Set the reference orientation based on the center coordinate point of the service area;

[0065] Set several virtual coordinate points on the ground Figure 3 dimensional space, based on virtual coordinate points on the ground Figure 3 The dimensional space is provided with a base map reference plane perpendicular to the reference orientation, wherein the virtual coordinate points correspond one to one with the service areas;

[0066] Several editing operation areas are set based on the base map reference plane, wherein the editing operation areas are used to load two-dimensional or three-dimensional materials;

[0067] Select the material corresponding to the editing operation area, adjust the material based on the relative relationship between the editing operation area and the center coordinate point of the service area, and set the playback timing of the material;

[0068] Push update information to terminal devices.

[0069] By adopting the above technical solution, the viewing area is usually an area suitable for tourists to view the scenery. Since the magnitude of the height difference among tourists is small, far smaller than the magnitude of the distant view, it can be approximately regarded as that tourists all use the same viewing height. Therefore, it is only necessary to demarcate the area at the corresponding position on the satellite two-dimensional map as the viewing area.

[0070] Since the viewing area has a certain horizontal length relative to the target, the viewing effect of the target will be different at different positions in the viewing area. If the same material is placed, misalignment and unreality will occur. The viewing area is divided into multiple service areas, and the service areas are separated by blank buffers. Since the service areas are relatively small, based on the principle of perspective, when viewing a large target from a distance, a small distance movement has little effect on the visual effect. Therefore, the visual effect of viewing the superimposed picture of the target and the virtual image in a single service area is close and relatively realistic. Since there is a certain interval between the service areas, the resulting linear dimension is sufficient to have a large impact on the visual effect, such as visual offset. Therefore, different service areas should have corresponding base map datums.

[0071] Because each viewing area has a different optimal viewing orientation, virtual coordinate points are designed for each viewing area's optimal viewing orientation. The basemap datum plane is then extended left and right based on these virtual coordinate points to achieve a good viewing experience. Based on this, an editing area is designed on the basemap datum plane to accommodate editable materials. Designers can adaptably edit and publish these materials, making the viewing experience more realistic for users within the service area.

[0072] When the editing is completed, the corresponding data generated will be packaged into an update package and pushed to the terminal device for downloading by the terminal device, or played in real time on the terminal device using a high-bandwidth cloud service.

[0073] In summary, this solution enables unified playback of the same scene for different terminal devices by presetting the timing, while adjusting the playback effects of resources in a large area, thereby achieving the spatial and temporal unification of the enhanced images of the tourism scene.

[0074] Optionally, the following steps are also included: entering gyroscope information corresponding to the base map reference plane and using it as preset gyroscope information, wherein the preset gyroscope information corresponds to a state in which the device's shooting direction is the reference orientation and is held horizontally.

[0075] By adopting this technical solution, when designing a basemap reference plane based on the viewing area, a basemap shot on-site is used to achieve optimal framing. The orientation of this basemap is set as the reference orientation, along with corresponding gyroscope information. During application, if the user terminal's real-time gyroscope information matches the preset gyroscope information, the terminal's playback screen will achieve the designed effect.

[0076] Optionally, the step of dividing the viewing area into a service area and a blank buffer area includes:

[0077] Enter the coordinates of the center of the service area, where the center of the service area is located within the viewing area;

[0078] Set the area within the preset radius from the center coordinate point of the service area as the service area;

[0079] Set the portion outside the service area within the viewing area as a blank buffer zone.

[0080] Optionally, the service area and the blank buffer area are arranged at intervals, and a ratio of the length of the service area to the length of the blank buffer area in the arrangement direction is greater than one.

[0081] Optionally, the step of setting a plurality of editing operation areas based on the base map reference plane includes:

[0082] An orientation reference point is set on the base map reference plane, and an editing operation area for placing two-dimensional materials is set based on the orientation reference point;

[0083] An anchor reference point is set on the base map datum plane, and an editing operation area for placing three-dimensional materials is set based on the anchor reference point.

[0084] Optionally, the editing operation area for placing two-dimensional materials is coplanar with or angled with the base map reference plane.

[0085] Optionally, the method of adjusting the material includes stretching, scaling, rotating and / or translating the material.

[0086] By adopting the above technical solution, 2D assets can be placed in the editing area and subjected to editing operations such as stretching, scaling, rotating, and / or translating. The editing area can overlap with the basemap reference plane or be at an angle to it. During use, the editing area can be rotated around the orientation reference point to adjust the orientation of the 2D asset. The orientation reference point is also used to position the editing area on the basemap reference plane, so that the editing area moves synchronously when the basemap reference plane is moved or rotated.

[0087] Three-dimensional materials can be placed in the editing operation area for editing operations such as stretching, scaling, rotating and / or translating. The anchor reference point is used to position the editing operation area on the base map reference plane so that the editing operation area can move synchronously when the base map reference plane moves or rotates.

[0088] In a third aspect, the present application provides an electronic device that adopts the following technical solution:

[0089] A virtual space interactive system based on AR technology, comprising:

[0090] The zoning acquisition module is used to obtain viewing area information, where the viewing area is divided into several service areas. Each service area corresponds to a base map reference surface containing location information. Points on the base map reference surface correspond to editing operation areas for placing materials.

[0091] A positioning module, configured to obtain positioning information and determine a working reference plane based on a correspondence between the positioning information and the service area, wherein the working reference plane is a base map reference plane corresponding to the positioning information;

[0092] The loading module is used to load materials into the editing operation area and adjust the materials and playback timing based on preset requirements;

[0093] An orientation module, used to determine the display orientation of the working reference surface based on the positioning information;

[0094] A matching module is used to obtain real-time on-site image information and match the working reference surface with the real-time on-site image information;

[0095] A mapping module is used to superimpose the display surface content of the visual operation area on the real-time on-site image information as output image information, wherein the visual operation area is the editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image;

[0096] The display module is used to display the superimposed image on the screen of the terminal device.

[0097] In a fourth aspect, the present application provides an electronic device that adopts the following technical solution:

[0098] An editing and updating system, comprising:

[0099] Zoning module, used to set the viewing area on the ground Figure 2 dimensional space, and divide the viewing area into service areas and blank buffer areas;

[0100] A direction setting module is used to set a reference direction based on the center coordinate point of the service area;

[0101] The benchmark setting module is used to set several virtual coordinate points on the ground. Figure 3dimensional space, based on virtual coordinate points on the ground Figure 3 The dimensional space is provided with a base map reference plane perpendicular to the reference orientation, wherein the virtual coordinate points correspond one to one with the service areas;

[0102] An editing area setting module is used to set a number of editing operation areas based on the base map reference plane, wherein the editing operation area is used to load two-dimensional or three-dimensional materials;

[0103] The editing module is used to select the material corresponding to the editing operation area and adjust the material based on the relative relationship between the editing operation area and the center coordinate point of the service area;

[0104] The push module is used to push update information to terminal devices.

[0105] In a fifth aspect, the present application provides an electronic device that adopts the following technical solution:

[0106] An electronic device, comprising:

[0107] one or more processors;

[0108] Memory;

[0109] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to:

[0110] Execute the above-mentioned virtual space interaction method based on AR technology.

[0111] In a sixth aspect, the present application provides a server that adopts the following technical solution:

[0112] A server, comprising:

[0113] one or more processors;

[0114] Memory;

[0115] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to:

[0116] Execute the edit update method described above.

[0117] In a seventh aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0118] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the above method.

[0119] The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement:

[0120] Such as the virtual space interaction method based on AR technology mentioned above;

[0121] and / or, the edit update method described above.

[0122] In summary, this application includes at least one of the following beneficial technical effects:

[0123] This invention breaks the constraints of time and space, perfectly integrating virtual space with real space; it is not limited by time and runs in real time 24 hours a day, 365 days a year;

[0124] The present invention requires on-site check-in and photo taking, bringing citizens from indoors to outdoors, and promoting the development of the real economy;

[0125] The present invention keeps pace with the times, can easily modify the display content, has low requirements for scenes and materials, and has good wide adaptability and scalability;

[0126] The present invention focuses on the production of virtual space content, with low investment and high output, which forms a sharp contrast with the large investment in real scene construction and has good economic value;

[0127] The current nighttime real-scene lighting consumes a huge amount of energy, and activities such as fireworks shows will cause strong environmental pollution and bring potential safety hazards. However, the present invention consumes almost zero energy at night and does not cause environmental pollution, which is green and environmentally friendly.

[0128] The present invention does not require additional physical equipment for support and can be quickly put into practical use with the help of existing terminal devices such as mobile phones and AR glasses. It is easy to operate and maintain in the future. A large number of extended services and innovative services are updated through the network, and the content becomes richer over time. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 Used to illustrate the steps of a virtual space interaction method based on AR technology in a certain embodiment of the present application.

[0130] Figure 2 It is used to illustrate the scene of the Oriental Pearl Tower and the Huangpu River used as an example in this application.

[0131] Figure 3 It is a satellite map used to illustrate the relative position relationship between the service area, viewing area, base map reference plane, and landscape in a certain embodiment of the present application, and to illustrate the reference orientation and the display orientation of the base map reference plane.

[0132] Figure 4 It is used to illustrate a method for dividing a viewing area into a service area and a blank buffer area in a certain embodiment of the present application.

[0133] Figure 5 Used to illustrate the method of setting the base map reference plane in a certain embodiment of the present application.

[0134] Figure 6 This is a real-life scene diagram used to illustrate the relationship between the base map reference plane, editing operation area, and position material in a certain embodiment of the present application.

[0135] Figure 7 Used to illustrate the sub-steps of S2 in a certain embodiment of this application.

[0136] Figure 8 Used to illustrate optional sub-steps of S2 in a certain embodiment of the present application.

[0137] Figure 9 Used to illustrate the sub-steps of S3 in a certain embodiment of this application.

[0138] FIG10 is used to illustrate the visual changes before and after moving from point A to point B in the service area of ​​this application.

[0139] FIG. 11 is used to illustrate the visual changes before and after moving from point A to point B in the related art.

[0140] Figure 12 Used to illustrate the sub-steps of S3 in another embodiment of the present application.

[0141] Figure 13 Used to illustrate the sub-steps of S4 in a certain embodiment of the present application.

[0142] Figure 14 Used to illustrate the sub-steps of S5 in a certain embodiment of the present application.

[0143] Figure 15 Used to illustrate the steps of an editing and updating method in a certain embodiment of the present application. DETAILED DESCRIPTION

[0144] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the inventive concepts. Some of the figures in the drawings of the present disclosure, which are part of this specification, represent structures and devices in block diagram form to avoid making the disclosed principles complicated and obscure. For the sake of clarity, not all features of an actual implementation are necessarily described. In addition, the language used in this disclosure has been selected primarily for readability and instructional purposes and may not have been selected to delineate or limit the subject matter of the invention, thereby resorting to the necessary claims to determine such inventive subject matter. References in this disclosure to "one embodiment" or "an embodiment" mean that the specific features, structures or characteristics described in conjunction with that embodiment are included in at least one embodiment, and multiple references to "one embodiment" or "an embodiment" should not be understood to necessarily all refer to the same embodiment.

[0145] Figure 1 FIG. 1 is a flowchart of a virtual space interaction method based on AR technology in a certain embodiment. It should be understood that although Figure 1-8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be executed in other orders; and Figure 1-8 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0146] In addition, the numbers of the steps in this embodiment are for convenience of explanation only and do not limit the execution order of the steps. In actual application, the execution order of the steps can be adjusted as needed, or performed simultaneously. These adjustments or replacements are all within the scope of protection of the present invention.

[0147] The terms "a," "an," and "the" are not intended to refer to singular entities, but include the general class of which a specific example can be used for illustration. The use of the terms "a" or "an" can mean any number of, including "one," "one or more," "at least one," and "one or more than one." The term "or" means any one of the alternatives, as well as any combination of the alternatives, including all of the alternatives, unless the alternatives are expressly indicated to be mutually exclusive. The phrase "at least one of' in combination with a list of items means that a single item from the list can be used, or any combination of the items from the list can be used, unless expressly indicated to the contrary.

[0148] At present, Augmented Reality (AR) is a new technology that integrates real world information and virtual world information "seamlessly", which is to simulate the entity information that is difficult to experience in a certain time and space range of the real world and then superimpose it on the real picture, and to be perceived by human senses, so as to achieve a sensory experience beyond reality. The real environment and the virtual object are superimposed in the same picture or space at the same time. It not only presents the information of the real world, but also displays the virtual information at the same time, and the two kinds of information complement and superimpose each other. The embodiments of the present application disclose a virtual space interaction method based on AR technology, which realizes multi-person interaction by using virtual space, and based on a unified real-virtual coexistence parallel world coordinate system at night, the online and offline are penetrated, the user interaction experience is formed, and the space internet is formed. By using the virtual space interaction technology, the tourists in the same viewing area can see the combination of the material playing picture and the actual image at the same time, for example, during the New Year countdown, the tourists can see the playing of the light show in the same viewing area through different terminal devices.

[0149] It should be noted that before the implementation of the virtual space interaction method, in addition to deploying the corresponding virtual space interaction system on the terminal device, it is also necessary to download the initial resource package or the resource update package from the server. In order to ensure that the versions of the resource update package used in the terminal devices of the users are consistent, in some embodiments, the system can perform update package version detection when the user opens, or require forced update, or update after reaching the preset location. As an example, in an embodiment, the resource package update can be performed based on the following steps a-c:

[0150] a. Detecting user settings, generating a request update input instruction based on the user settings or generating a timing trigger instruction based on the user settings, wherein the request update input instruction is generated based on the input behavior of the user.

[0151] b. Sending an update detection instruction to the server based on the timing trigger instruction or the request update input instruction;

[0152] Users can set the system to either automatic update mode or manual update mode. In automatic update mode, the system can generate a timed trigger instruction, such as generating an update detection instruction every day, and sending it to the server for version detection to perform a version update. In manual update mode, the system can receive a trigger instruction input by the user, such as manually touching a designated interactive button on a mobile phone APP, and then send an update detection instruction to the server based on the trigger instruction. In some embodiments, the location of the terminal device can also be obtained based on positioning information, and after entering a designated area, a request for update input instruction is triggered to be sent to the server for version detection.

[0153] c. Request the server to download the resource package based on the update reply instruction returned by the server.

[0154] The initial resource package or resource update package contains material files, placement parameter files, timing files, etc. The material file is the material image file, which can be a two-dimensional image file or a three-dimensional image file according to the specific type. The placement parameter file contains parameter information such as the placement direction and scaling ratio of the material. The timing file contains the playback time sequence of the material, which strictly corresponds to the international standard time, so the terminal needs to be calibrated before use. Of course, in some embodiments, the timing can also strictly correspond to other clocks, as long as it corresponds to the same clock. After the server releases a new resource update package, in different embodiments, the old resource update package in the terminal device can be deleted, replaced with the corresponding file to save storage space, or retained for subsequent review. Reference Figure 1 , the virtual space interaction method comprises the following steps:

[0155] S1. Obtain viewing area information.

[0156] While urban tourist destinations offer an ever-increasing array of options, the allure of landmark buildings and landmark areas remains undiminished. For example, the Forbidden City and Tiananmen Square in Beijing, the Bund and Oriental Pearl Tower in Shanghai, and the Pearl River and Canton Tower in Guangzhou are all must-see destinations for tourists. This type of landmark building is characterized by large volume, and the landmark area is characterized by a large area, such as The Tiananmen Gate Tower is 66 meters long, 37 meters wide and 32 meters high. The best places to take photos are in the middle and far areas of Tiananmen Square. Take Tiananmen Square and the night sky into the camera. Figure 2 and Figure 3 The Oriental Pearl Tower is 468 meters high, a good place to take photos. The section is located on the Bund by the Huangpu River.

[0157] Therefore, in this solution, such areas can be selected as viewing areas to provide subsequent services. Specifically, the location-based service (LBS) is abbreviated as LBS. Its main working principle is to use radio communication networks or GPS and other positioning methods to determine the location of the mobile device, and then provide corresponding value-added services based on the user's location information.

[0158] It's important to note that this solution's application scenarios aren't limited to checking in and photographing city landmarks. It can also be used to designate viewing areas for mountains, rivers, the starry sky, and even non-landmark areas. Furthermore, this solution can also be applied to indoor areas, as long as the device can be accurately positioned.

[0159] The division of the viewing area can be achieved through a variety of methods. In different embodiments, the division can be based on base station signal strength, or it can be based on signal detection by physical communication equipment installed on site. Any method that can accurately locate the position on the viewing area is acceptable. For example, in one embodiment, the viewing area is set in a wide outdoor scene, and a set of coordinates of a designated area is delineated on a plane coordinate interval established at the location based on a satellite positioning system as the viewing area. For example, the area along the Huangpu River on the Bund is designated as the viewing area.

[0160] Since the viewing area has a certain extension in a certain direction, the viewing effect of the target at different positions will be different. If the same material is placed in the same position, misalignment and unreality will occur. Continuing with the example of the area along the Huangpu River on the Bund, the span of this area is several thousand meters, and the scenes of the Oriental Pearl Tower seen at both ends of this area are significantly different. Based on this situation, in some embodiments of the present scheme, the viewing area can be divided into several service areas, and the service areas are set at intervals and blank buffer zones are set between adjacent service areas. The service areas are used to provide virtual space interactive services based on AR technology, while blank buffer zones are not provided. The size of the service area and the blank buffer zone is affected by the size and distance of the target. From the perspective principle, the larger and farther the target is, the smaller the impact of the observer's movement of a fixed distance on the observation effect. For example, referring to Figure 3 , Figure 3 The frame lines are only used to indicate relative position relationships, not to indicate size relationships. The area along the Huangpu River on the Bund is the viewing area, which is Figure 3 The black box on the left side of the center image is shown. The service area is 50 meters long along the river, roughly the area within the circle in the black box. The blank buffer zone is 500 meters long along the river, roughly the area between the black box and the circle in the box. Therefore, the observation effect of the Oriental Pearl Tower within a single service area is roughly the same.

[0161] It should be noted that the viewing area is not necessarily divided into a service area and a blank buffer zone. In some embodiments, the viewing area can have only a service area without a blank buffer zone. This division method is suitable for small landscapes with a small viewing area. In special cases, this division method can also be used for extremely large landscapes with a lack of reference objects, such as the sky and the sea.

[0162] Specifically, refer to Figure 4 , a method for dividing the service area and blank buffer zone in the viewing area includes the following steps:

[0163] S111. Enter the center coordinate point of the service area, where the center coordinate point of the service area is located within the viewing area.

[0164] S112. Set the area within a preset radius from the center coordinate point of the service area as the service area.

[0165] S113. Set the portion of the viewing area outside the service area as a blank buffer zone.

[0166] It should be noted that the shape of the service area in this solution is not limited to a circle, and any service area design shape that can easily determine whether to leave or enter the service area is acceptable.

[0167] Each service area corresponds to a base map reference surface, which is used as an intermediate reference connecting the real scene and the virtual scene. The materials that need to be superimposed on the real scene photos are anchored on the base map reference surface. Since different service areas correspond to different observation effects, for example, the observation positions of the same material in different service areas should be different, each service area corresponds to a base map reference surface. In this embodiment, the preset radius of the Bund service area is set to fifty meters. Specifically, refer to Figure 5 , the base map reference surface can be set based on the following methods:

[0168] S121. Photograph a base map at the center coordinate point of the service area toward the target landscape, and simultaneously obtain corresponding gyroscope information as preset gyroscope information.

[0169] S122. Calculate the shooting direction corresponding to the base map based on the center coordinate point of the service area and the preset gyroscope information and use it as the reference orientation.

[0170] S123. Take a point on a ray emitted from the center coordinate point of the service area along the reference direction as a virtual coordinate point. The virtual coordinate point is far away from the service area relative to the target landscape.

[0171] S124. Generate a base map reference surface based on the left and right extension of the virtual coordinate point, and set the orientation of the base map reference surface as the display orientation. Relative to the center coordinate point of the service area, the display orientation is opposite to the reference orientation.

[0172] The basemap datum isn't set in silos; it's based on virtual coordinates that reflect the basemap datum's location. For example, by setting a virtual coordinate point with the Oriental Pearl Tower facing away from the Bund, the basemap datum as seen from the Bund will be behind the Oriental Pearl Tower. The virtual coordinate point can be set much further away from the Oriental Pearl Tower, resulting in the basemap datum appearing farther away from the Oriental Pearl Tower.

[0173] In other words, the base map datum and the service area center coordinates are both based on the ground. Figure 3 In a 3D space, any point in this space can be represented by corresponding coordinates. Since the position, size, and orientation of materials may vary depending on the viewing screen, maintaining them if they are all fixed to the basemap reference plane would be difficult. Therefore, the basemap reference plane can be used to set up editing operation areas for placing materials based on points on the surface. When the basemap reference plane moves or rotates, the editing operation area will also move synchronously with the basemap reference plane.

[0174] Here you can use Figure 3 Provide explanations, Figure 2 The black triangle in the figure represents the Oriental Pearl Tower. The dotted arrow indicates the reference orientation of the service area. The solid black line pointed to by the dotted arrow represents the base map reference plane. The direction indicated by the dotted arrow is perpendicular to the solid black line, and the foot of the perpendicular is the virtual coordinate point. The small black arrow at the top indicates the display orientation of the base map reference plane.

[0175] Specifically, in some embodiments, the editing operation area is a two-dimensional plane with an orientation reference point. This two-dimensional plane is coplanar with or angled with the base map reference plane, and the orientation reference point is located on the base map reference plane. In other embodiments, the editing operation area is a three-dimensional space with an anchor reference point, and the anchor reference point is located on the base map reference plane. Of course, the base map reference plane can also contain both types of editing operation areas.

[0176] 2D assets can be placed in the editing area for editing operations such as stretching, scaling, rotating, and / or translating. The editing area can overlap with the basemap reference plane or be at an angle to it. The editing area can be rotated around an orientation reference point to adjust the orientation of the 2D asset. The orientation reference point also serves to position the editing area on the basemap reference plane, ensuring that the editing area moves in sync with the basemap reference plane's movement or rotation.

[0177] Three-dimensional materials can be placed in the editing operation area for editing operations such as stretching, scaling, rotating and / or translating. The anchor reference point is used to position the editing operation area on the base map reference plane so that the editing operation area can move synchronously when the base map reference plane moves or rotates.

[0178] Specifically, the types of materials are diverse, that is, laser shows, light and shadow shows, projection shows, drone shows, fireworks shows, etc., which can be performed with the city night scene light show; they can also be star patterns, aurora patterns, snowflake patterns, meteor patterns, cherry blossom patterns, maple leaf patterns; they can also be commercial releases, interactive advertisements, personal creative exhibitions, creating scenes that are usually not seen in the city night sky; or they can be an organic combination of patterns or elements to make meaningful expressions. Figure 6 As shown, it is the actual shooting of fireworks show material effect, coarse right angle box for the bottom of the reference surface, the fine round box for the editing operation area, the "Happy New Year" in the editing operation area is the material, the area of the person is the service area. Figure 6 For showing the material placement effect, the position relationship between the person and the landscape, the position relationship between the person and the bottom reference surface, the position relationship between the bottom reference surface and the editing operation area, without limiting the size of the bottom reference surface and the editing operation area.

[0179] It should also be noted here that the design of the bottom reference surface reduces the maintenance cost of the whole system. It can be easily thought that several fixed points are defined in space and the editing operation area is designed on the fixed points to place the material. This will bring a problem that the placement direction of the two-dimensional material and the three-dimensional material in each editing operation area is difficult to determine during the later maintenance, and it is difficult to uniformly adjust the material in the editing operation area, such as translation or rotation. In addition, directly designing the editing operation area depending on the fixed point in space, the selection of the fixed point is actually not intuitive, and the actual maintenance personnel needs to repeatedly debug to ensure that the material in it has good visual effect, which not only wastes time and effort, but also needs to be debugged on site every time a new fixed point is designed, and the effect is not satisfactory and the coordination is poor.

[0180] Therefore, the design of the bottom reference surface in the present scheme can give the later maintenance personnel an intuitive sense of space based on a standard plane, even if the editing operation area extends outward on the bottom reference surface, the spatial relative relationship can be clearly displayed, and the maintenance efficiency of the personnel is greatly improved. In addition, by operating the bottom reference surface, the material in the editing operation area is indirectly uniformly operated, which can simultaneously reduce the spatial assistance degree and time complexity of the algorithm compared with the scheme of designing the editing operation area on the spatial fixed point, and improve the running efficiency.

[0181] After the acquisition of the viewing area information in step S1 is completed, step S2 is entered.

[0182] S2. Obtain positioning information, and determine a working reference surface based on the correspondence between the positioning information and the service area, wherein the working reference surface is a bottom reference surface corresponding to the positioning information.

[0183] The terminal device needs to be equipped with a Bluetooth module, a cellular communication module or a satellite positioning module, as long as it has the function of accurately positioning the position of the terminal device. The positioning information is used to determine whether the terminal device is in the service area, which service area it is in, and the position in the service area. When the terminal device is determined to be in the service area, the corresponding working reference surface of the service area can be obtained through the database query. After obtaining the working service area, the material information and the corresponding preset material playing time sequence of the editing operation area on the working reference surface can be retrieved accordingly.

[0184] Specifically, refer to Figure 7 In some embodiments, S2 includes the following steps:

[0185] S201. Obtain real-time satellite positioning coordinate information, and determine the service area based on the correlation between the distance difference between the satellite positioning coordinate point and the center coordinate point of each service area and the preset radius; wherein the center coordinate of the service area is the center coordinate of the service area, and the preset radius is the radius length of the service area.

[0186] S202. Controlling the camera module to be turned on and off based on the relative position relationship between the satellite positioning coordinate point and the service area.

[0187] For example, S202 can be implemented by the following steps: when the distance difference between the satellite positioning coordinates and the service area center coordinates is greater than a preset radius, the camera module is controlled to be turned off; when the distance difference between the satellite positioning coordinates and the service area center coordinates is less than the preset radius, the camera module is controlled to be turned on. The preset radius is pre-set in the background, and the relevant parameters are included in the initial resource package or resource update package.

[0188] S203. Obtain the base map datum corresponding to the service area and load it as the working datum.

[0189] Through S202, when the user leaves the service area, the camera module's closing action can serve as a reminder to the user. At the same time, the opening and closing of the camera module can also guide the user in finding and entering the service area. For example, if the radius of the service area is 50 meters, when the user approaches the satellite positioning coordinate point 49 meters away, the camera turns on and the phone screen changes to a real-time image, reminding the user that they have entered the service area. When the user is 51 meters away from the satellite positioning coordinate point, the camera turns off, reminding the user that they have left the service area. It should be noted that there is no strict order restriction for S202 and S203, and these steps can be performed in any order.

[0190] It should be noted that the imaging of the camera module of the terminal device is formed by the lens capturing the real light and shadow, which has a very high response speed, and the high refresh rate of the imaging of the terminal when moving is sufficient to make the human eye unable to observe the jumping of the adjacent frames of the picture. When the bandwidth of the terminal device is sufficient, the terminal device can perform high-frequency and high-precision satellite positioning, so that the terminal device can adjust the display orientation of the working reference surface once every moment when moving. As long as the refresh rate is high enough, multiple subtle adjustments can form a continuous adjustment process, so that the material movement of the picture on the reference surface is smooth and continuous. However, in the current common technology, the accuracy of satellite positioning or other positioning methods is not high enough, and the refresh frequency of the positioning information is not fast enough. When the terminal device receives a positioning information during movement, the system processes it and loads the material image on the working reference surface and fuses it to the real-time live image. First, the material image is lagging behind and mismatched with respect to the real-time live image. Second, due to the slow refresh frequency of the positioning information, the change of the material image is jumping and not smooth. Third, when the satellite positioning is sensitive enough, it will drift due to environmental interference and other factors, resulting in picture mismatch. These factors will significantly affect the user experience.

[0191] Therefore, optionally, with reference to Figure 8 When it is determined that the terminal device is located in the service area, the S2 can further include the following sub-steps:

[0192] S211. Obtain the analog positioning point information in the service area, wherein a plurality of analog positioning points are distributed in each service area.

[0193] The analog positioning points can be uniformly arranged in the service area or non-uniformly arranged in the service area. The satellite positioning coordinates of the analog positioning points can be obtained by field sampling or based on the center coordinates of the service area plus or minus the distance. However, it should be noted that the distribution of the analog positioning points in the server cannot be too dense, and can be adaptively adjusted based on the current satellite positioning accuracy. As an example, the spacing of the analog positioning points in this scheme is greater than two meters.

[0194] S212. Obtain the analog positioning point closest to the satellite positioning coordinate point and use it as the terminal position coordinate point.

[0195] Since the terminal position coordinate point is positioned at the same coordinate point, even if the refresh frequency of the positioning information of the terminal device is low, the material image presented is fixed and does not jump.

[0196] Correspondingly, in the subsequent steps of the embodiments corresponding to S211 and S212, the satellite positioning coordinate point can be replaced by the terminal position coordinate point.

[0197] After the determination of the working reference plane in S2, the process proceeds to S3.

[0198] S3. Determine the display orientation of the working reference plane based on the positioning information to determine the display plane.

[0199] As can be seen from S124, the display orientation of the working reference plane is opposite to the reference orientation with respect to the center coordinate point of the service area. It should be noted that the reference orientation is a fixed orientation, while the display orientation is an adjustable parameter.

[0200] Since the base map reference plane is formed in a three-dimensional space, the placement of the materials in the editing operation area is not actually limited to the two-dimensional space of the base map reference plane, that is, the entire base map reference plane and the entire editing operation area correspond to a three-dimensional model with a 720° panoramic view. The editing operation area can also extend in other directions, such as extending below the ground, extending into the service area, and placing materials in the extended area. The solutions provided by the present application can achieve these effects. In fact, material image pictures can be obtained without obstacles from all areas except the non-viewing area. However, since the arrangement of the editing operation area on the base map reference plane has the significance of enabling the materials to be reasonably arranged in a facing direction to achieve the best viewing effect, during the design, a display plane will be determined based on the position of the service area. This display plane is formed by the material positions of the editing operation area on the working reference plane, that is, it corresponds to the display orientation of the working reference plane.

[0201] Optionally, in different embodiments, the display plane corresponds to the material images that can be seen within a 80° viewing angle or within a 120° viewing angle of the horizontal plane of the center coordinate point of the service area. The relevant parameter files can be pre-set as needed and included in the update package.

[0202] As mentioned above, the visual effects of the superimposed picture of the viewing target and the virtual image are close in a single service area, but there are still some differences in the playing picture of the long shot material viewed at different positions in the same service area, and the differences are smaller when the positions are closer to the middle of the base map reference surface and are larger when the positions are farther from the middle of the base map reference surface. Here, the screen of a cinema can be used for analogy. Since the range of the base map reference surface is large and occupies a large visual angle relative to a person, when the person faces the base map reference surface, it is equivalent to sitting in the front row of seats of the cinema to watch the giant screen. For the audience in the middle of the front row of seats, the effect of watching the middle picture of the screen is the best, and watching the pictures on the two sides of the screen will cause a certain visual distortion, which is determined by the mechanism of using the human eye to watch the two-dimensional picture to simulate the three-dimensional vision. For the audience on the left of the front row of seats, the distortion is greatly enhanced when watching the picture on the right of the screen, and the distortion is serious. The audience on the right of the front row of seats also has the same problem. Therefore, in order to compensate for the visual differences, the present scheme can make adaptive adjustment to the display orientation based on the relative relationship between the reference orientation information and the real-time orientation information, so that the user can clearly see the middle of the base map reference surface at different positions.

[0203] Based on this, in order to achieve the purpose, S3 can be further limited in some embodiments as follows:

[0204] Adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information; wherein the reference orientation information is the orientation information of the service area center coordinate point to the virtual coordinate point, the real-time orientation information is the orientation information of the terminal position coordinate point to the virtual coordinate point, and the virtual coordinate point is the reference point for establishing the base map reference surface.

[0205] Since the reference orientation and the real-time orientation have a certain angle, the coordinates of each point on the working reference surface are rotated by a corresponding angle, so that the working reference surface corresponds to the real-time orientation. For example, if the virtual coordinate point is taken as the reference center, if the distance of 0.1° central angle relative to the virtual coordinate point is moved in the service area, the working reference surface is synchronously rotated by a corresponding angle to ensure that the display orientation is the direction from the virtual coordinate point to the terminal satellite positioning coordinate point.

[0206] Since satellite positioning is introduced to improve the problem of visual distortion, it will undoubtedly cause a problem. When the satellite positioning is sensitive enough, it will drift due to environmental interference and other factors, resulting in mismatch of the picture and causing significant impact on the use experience.

[0207] Therefore, in order to solve this problem, optionally, referring to Figure 9 In some embodiments, S3 includes the implementation of S311-S313.

[0208] S311. Obtain the simulation positioning point information in the service area, wherein a plurality of simulation positioning points are distributed in each service area.

[0209] The principle of this step is similar to the scheme of setting simulation positioning points in the service area in the above-mentioned S211 and S212 steps. The service area has a certain area, and the simulation positioning points can be uniformly set in the service area or non-uniformly set in the service area. The satellite positioning coordinates of the simulation positioning points can be obtained by field sampling or based on the center coordinates of the service area plus or minus the distance.

[0210] S312. Obtain the simulation positioning point closest to the satellite positioning coordinate point and take it as the terminal position coordinate point.

[0211] S313. Read the preset orientation information corresponding to the terminal position coordinate point to determine the display orientation of the working reference surface, wherein the preset orientation information is the preset orientation of the base map reference surface to the virtual coordinate point.

[0212] Due to the high sensitivity of satellite positioning, when interference is encountered, it is easy to move in a small range. When the user position drifts, it is actually still moving near the simulation positioning point. Therefore, the simulation positioning point plays an anchoring role. The small range near the simulation positioning point adopts the preset orientation corresponding to the simulation positioning point. At the same time, since the terminal position coordinate point is positioned on the same coordinate point, even if the positioning information refresh frequency of the terminal device is low, the material image presented is fixed and will not jump. Specifically, referring to FIG. 10, the hollow square box is used to illustrate the service area at the Bund, the large circle in the hollow square box is used to illustrate the view area, points A and B are used to illustrate two different simulation positioning points, and the two curved lines are used to illustrate the Huangpu River. The triangle is used to illustrate the Oriental Pearl Tower, the dashed lines extending from points A and B are used to illustrate the display orientation of the working reference surface corresponding to the points, and the focal points of the dashed lines extending from points A and B are virtual positioning points on the working reference surface. The gray solid small square is used to illustrate the same editing operation area and the material inside it. The relationship between the two editing operation areas is the relationship before and after the rotation relative to the virtual positioning point. From Figure 10a It can be seen that after the terminal device moves from point A to point B, the position of the editing operation area is approximately unchanged, which is manifested as and are close to and are close to 0. From Figure 10b It can be seen that the viewing angles of points A and B to the Oriental Pearl Tower and the materials in the editing operation area are almost unchanged, which are manifested as and are almost equal.

[0213] Since the display orientation corresponding to the preset orientation is pre-set, there is no need to recalculate and adjust the working reference surface in the background, so it has the advantages of saving computing power and fast response speed.

[0214] In addition, it is assumed that the implementation of this solution is based on the related technology 2, that is, without using the positioning system, the real-time scene image is directly mapped. This solution seems to be able to solve the problem of picture jumping, but the visual effect produced is very different. For example, referring to Figure 11a and 11b The features in the figure are the re-use of Figure 10, the difference is that the relative position of the reference surface and the position of the observation point are fixed. Figure 11a It can be seen that the position of the editing operation area changes greatly, as shown in Larger. Figure 11b It can be seen that the viewing angles of the Oriental Pearl Tower and the materials in the editing area vary greatly between points A and B, as shown in the following figure: and The actual visual effect is that the fireworks pattern blooming at point A is quite far from the Oriental Pearl Tower. When it moves a few dozen meters to point B, the fireworks pattern is completely blocked by the Oriental Pearl Tower, or the fireworks pattern is blocked in front of the Oriental Pearl Tower, which gives users a clear sense of unreality.

[0215] In other embodiments, referring to Figure 12 , S3 includes sub-steps S321-S324 to implement:

[0216] S321. Obtain real-time gyroscope information, and determine whether the device enters or exits the raised state based on the real-time gyroscope information.

[0217] S322. Obtain the satellite positioning coordinate point corresponding to the moment the device enters the raised state and lock it as the terminal position coordinate point.

[0218] S323. Calculate the angle between the real-time orientation and the reference orientation, and adjust the display orientation of the working reference surface based on the angle.

[0219] S324. Unlock the terminal position coordinate point when the device exits the raised state.

[0220] To ensure the best possible photo or viewing experience, users typically remain still or make small adjustments when raising their phone to shoot. Therefore, when the device is detected as being raised, the satellite positioning coordinates are locked to prevent the material's position from jumping due to adaptive adjustments to the display orientation of the working reference surface during small movements. When the user adjusts the position, they typically lower the device. Therefore, when the device is released from the raised state, the terminal's position coordinates are locked to facilitate the acquisition of real-time satellite positioning coordinates for the next raised action.

[0221] Since satellite positioning is highly sensitive, it is easy to move in a small range when encountering interference. Therefore, in S322, the following steps can be used to determine the difference between drift interference and actual movement.

[0222] S3221. Calculate the distance difference between the real-time satellite positioning coordinates and the terminal position coordinate point as the real-time drift distance.

[0223] S3222. Update the terminal position coordinate point based on the relative size of the real-time drift distance and the preset drift error distance, wherein the updated value of the terminal position coordinate point is the satellite positioning coordinate corresponding to when the real-time drift distance exceeds the preset drift error.

[0224] By setting the drift range, the terminal position coordinate point within the error range can be continuously locked at the same point, greatly reducing the impact of positioning drift on the display orientation selection. On the other hand, in fact, the impact of the long-range visual effect produced by the slight displacement of the device is also very small, so this solution can greatly improve the user experience. In addition, optionally, a preset time threshold can also be introduced in S3222. Specifically, S3222 can be implemented through the following steps:

[0225] S32221. Get real-time drift distance;

[0226] S32222. Determine the relative size of the real-time drift distance and the preset drift error distance. If the real-time drift distance is less than the preset drift error distance, return to the previous step; if the real-time drift error distance is greater than or equal to the preset drift error distance, proceed to the next step;

[0227] S32223. Determine whether the duration during which the real-time drift error distance is greater than or equal to the preset drift error distance is greater than the preset time threshold. If not, return to S32221. If so, update the terminal position coordinate point. The updated value of the terminal position coordinate point is the current satellite positioning coordinate value.

[0228] For example, if the real-time drift error distance is greater than or equal to the preset drift error distance for more than one second, the terminal's position coordinates can be updated. The preset time threshold can be 0.1s, 0.2s, 0.5s, or other durations, and can be adaptively adjusted based on actual conditions.

[0229] S4. Acquire real-time on-site image information and match the working reference surface with the real-time on-site image information.

[0230] By superimposing the real-time image and the working reference surface, the material playback screen will be combined with the actual image, so that the real environment and virtual objects are superimposed on the same screen or space in real time. However, since the shooting posture of the terminal device is likely to be different from the shooting posture of the device when the base map is obtained, it is necessary to coordinate the working reference surface and the real-time scene image. Therefore, refer to Figure 13 , step S4 can be implemented by the following steps:

[0231] S401. Acquire real-time on-site image information and real-time gyroscope information, wherein the real-time on-site image information and the real-time gyroscope information correspond to each other.

[0232] S402. Obtain preset gyroscope information corresponding to the working reference surface, and rotate the working reference surface in real time based on the relative relationship between the preset gyroscope information and the real-time gyroscope information.

[0233] S403. Control the scaling of the material in the editing operation area on the working reference surface based on the camera module parameters.

[0234] The base map reference plane is initially set based on the base map, and the preset gyroscope information is recorded corresponding to it, that is, the posture of the base map reference plane should correspond to the gyroscope information. During shooting, the pitch angle and left and right orientation of the device are different. Therefore, it is necessary to determine the device posture when shooting real-time images based on the real-time gyroscope information. By using the relative relationship between the preset gyroscope information and the real-time gyroscope information, the working reference plane can be adjusted to a posture that matches the real-time picture. In addition, since the camera module parameters of different devices are different, such as wide angle, etc., the material is scaled based on the camera module parameters so that the image content obtained by different camera modules can correspond to the material position, ensuring that the material playback screen and the real-time image content on different devices can be relatively positioned.

[0235] In this solution, a transformation matrix can be obtained from the relative relationship between the preset gyroscope information and the real-time gyroscope information, and the initial base map reference plane is transformed through the transformation matrix to be converted into a working reference plane corresponding to the terminal position coordinate point.

[0236] S5. Load the material into the editing area and adjust the material and playback timing based on the preset requirements.

[0237] After the work is done, the corresponding material information and preset playback sequence for the editing operation area can be obtained based on the current updated resource package. Different users, when using different terminals, will see the same timing sequence of the materials even if they are in different service areas.

[0238] It should be noted that the offline manual work and the online manual work are separated by setting the two levels of the base map reference surface and the editing operation area. As shown in S121-S124, the setting of the base map reference surface needs manual work to take pictures on site to obtain the base map, determine the base map reference surface based on the shooting direction and the shooting position coordinates, and divide the area based on the base map to determine the position of the editing operation area to be set on the base map reference surface. These works are time-consuming but only need to be done once to complete the collection of parameters of the base map reference surface and the editing operation area. Based on the collected parameters, the design can be reused. The placement of the materials needs to be changed in each update, and manual work is required to place and modify them in each period, which is a long-term work, but only needs to be performed online.

[0239] Therefore, by the double-layer design of the base map reference surface and the editing operation area, the separable repetitive work is completed once, and manual parameter collection for repeated setting of the base map reference surface is not required, which greatly improves the work efficiency and manages the material information corresponding to multiple service areas with limited resources.

[0240] Specifically, referring to Figure 14 In an embodiment, step S5 includes the following sub-steps:

[0241] S501. Obtain the material placement information corresponding to the editing operation area on the working reference surface, and load the material in the editing operation area based on the material placement information, wherein the material placement information is used to achieve the preset requirements, including material files, material placement directions, and / or material scale information.

[0242] S502. Adjust the placement angle, color, and / or size of the material in the editing operation area based on the material placement information.

[0243] The initial resource package or the updated resource package published by the server contains material files, placement parameter files, and timing files. The material file is a material image file, which can be a two-dimensional image file or a three-dimensional image file according to the specific type. The placement parameter file contains parameters such as the placement direction and the scaling ratio of the material corresponding to each editing operation area. The timing file contains the playing time sequence of the material corresponding to each editing operation area. The timing strictly corresponds to the international standard time, so the terminal needs to be time-adjusted when used.

[0244] Since the same material can be used on different base map reference surfaces, and these materials have different placement angles and sizes, in order to reduce the data transmission amount and the space occupation amount, when loading the material onto the corresponding editing operation area, the material can be edited by stretching, scaling, rotating, and / or translating based on the preset material placement information to adapt to the placement requirements of the corresponding base map reference surface.

[0245] S503. Set the playback timing of the material based on the timing file.

[0246] The timing file sets the playback time sequence of the materials on each editing operation area. When loading the materials into the editing operation area, it is necessary to further set the playback timing of the materials according to the timing file to ensure that the material playback order of all terminals in the same service area is the same.

[0247] In some embodiments, the content is a moving image, such as a drone animation, with a specific playback duration, typically set to start and end at a specific time, or to loop. The preset playback sequence file updated on each terminal device is the same, so the playback effect of the content viewed on different terminal devices is consistent.

[0248] After the loading and playing of the material is performed, step S6 may be entered.

[0249] S6. Overlaying the display surface content of the visual operation area as output image information on the real-time on-site image information, wherein the visual operation area is an editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image.

[0250] Since the editing operation area may be a two-dimensional plane or a three-dimensional space, in some embodiments, the center of the terminal position coordinate point can be first projected onto the working reference surface to form an intermediate transition two-dimensional plane material, and then the center of the working reference surface can be projected onto the real-time image, thereby achieving superposition and output of the two. In other embodiments, the material in the editing operation area can be directly centrally projected or horizontally projected onto the working reference surface, thereby achieving superposition and output of the two. In addition, it should be noted that since the lens's framing range is difficult to cover the entire base map datum surface, the editing operation area corresponding to the lens's framing range is a visual operation area, and it is only necessary to map the material playback effect in the visual operation area onto the real-time image.

[0251] S7. Display the superimposed image on the screen of the terminal device.

[0252] The superimposed image is displayed on the screen of the terminal device, and finally the virtual and real space superimposed based on the camera function is fused and viewed in the viewing area. In different embodiments, the terminal device can be different, and the corresponding superimposed image display method can also be different. For example, for a mobile phone terminal, the present scheme can be realized through a native application program, and the screen and camera are directly called from the hardware level through the native program, so that the real-time live image obtained by the camera is fused with the output image information, and displayed on the screen. In another embodiment, the present scheme can also call the system camera program, and superimpose the real-time live image output to the screen by the camera program, so as to display the fused view on the screen. Or in the embodiment in which the terminal device is AR glasses, the system obtains the real-time live image by calling the camera module, and then calls the display module on the AR glasses to display the superimposed image. Any method that can display the superimposed image on the screen of the terminal device is within the protection scope of the present scheme.

[0253] In addition, the present application also discloses an editing updating method, which is applied to a server, refers to Figure 15 , and includes the following steps:

[0254] T1. Set a viewing area in a three-dimensional space, and divide a service area and a blank buffer area in the viewing area. Figure 2

[0255] T2. Set a reference direction based on the center coordinate point of the service area.

[0256] T3. Set a plurality of virtual coordinate points in the three-dimensional space, and set a bottom map reference surface perpendicular to the reference direction in the two-dimensional space based on the virtual coordinate points, wherein the virtual coordinate points correspond to the service area one by one. Figure 3 Figure 3

[0257] T4. Set a plurality of editing operation areas based on the bottom map reference surface, wherein the editing operation areas are used to load two-dimensional or three-dimensional materials. The method for adjusting the materials includes stretching, scaling, rotating and / or translating the materials, and the editing operation area used to place the two-dimensional materials is coplanar with or forms an angle with the bottom map reference surface.

[0258] Specifically, T4 includes the following steps:

[0259] T41. Set a direction adjusting reference point on the bottom map reference surface, and set an editing operation area used to place the two-dimensional materials based on the direction adjusting reference point.

[0260] T42. Set an anchoring reference point on the bottom map reference surface, and set an editing operation area used to place the three-dimensional materials based on the anchoring reference point.

[0261] ​​​T5. Select the material corresponding to the editing operation area, and adjust the material and set the playback timing of the material based on the relative relationship between the editing operation area and the center coordinate point of the service area.

[0262] T6. Push update information to the terminal device.

[0263] In addition, this application also discloses a virtual space interactive system based on AR technology, including:

[0264] The zoning acquisition module is used to obtain viewing area information, where the viewing area is divided into several service areas. Each service area corresponds to a base map reference surface containing location information. Points on the base map reference surface correspond to editing operation areas for placing materials.

[0265] A positioning module, configured to obtain positioning information and determine a working reference plane based on a correspondence between the positioning information and the service area, wherein the working reference plane is a base map reference plane corresponding to the positioning information;

[0266] The loading module is used to load materials into the editing operation area and play them based on the preset timing;

[0267] An orientation module, used to determine the display orientation of the working reference surface based on the positioning information;

[0268] A matching module is used to obtain real-time on-site image information and match the working reference surface with the real-time on-site image information;

[0269] A mapping module is used to superimpose the display surface content of the visual operation area on the real-time on-site image information as output image information, wherein the visual operation area is the editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image;

[0270] The display module is used to display the superimposed image on the screen of the terminal device.

[0271] In addition, this application also discloses an editing and updating system, including:

[0272] Zoning module, used to set the viewing area on the ground Figure 2 dimensional space, and divide the viewing area into service areas and blank buffer areas;

[0273] A direction setting module is used to set a reference direction based on the center coordinate point of the service area;

[0274] The benchmark setting module is used to set several virtual coordinate points on the ground. Figure 3 dimensional space, based on virtual coordinate points on the ground Figure 3 The dimensional space is provided with a base map reference plane perpendicular to the reference orientation, wherein the virtual coordinate points correspond one to one with the service areas;

[0275] The editing area setting module is configured to set a plurality of editing operation areas based on the bottom picture reference surface, wherein the editing operation area is configured to load two-dimensional or three-dimensional materials;

[0276] The editing module is configured to select the material corresponding to the editing operation area, and adjust the material based on the relative relationship between the editing operation area and the center coordinate point of the service area.

[0277] The pushing module is configured to push the update information to the terminal device.

[0278] The embodiment of the application further discloses an electronic device including a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to execute the virtual space interaction method based on the AR technology. The execution subject of the embodiment method can be a control device arranged on the electronic device. The current device can be an electronic device with WIFI function such as a mobile phone, a tablet computer, a notebook computer, etc. The execution subject of the embodiment method can also be a CPU (central processing unit) of the electronic device.

[0279] The embodiment of the application further discloses a server including a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to execute the editing and updating method. The execution subject of the embodiment method can be a control device arranged on the server. The current device can be an electronic device such as a workstation, a supercomputer, etc. The execution subject of the embodiment method can also be a CPU (central processing unit) of the electronic device.

[0280] The embodiment of the application further discloses a computer readable storage medium storing a computer program capable of being loaded and executed by the processor to execute the virtual space interaction method based on the AR technology. Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiment method can be realized by software and a general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on this understanding, the technical solution of the application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) and includes a plurality of instructions for making a device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) execute the method of each embodiment of the application.

[0281] The embodiment of the present application also discloses a computer-readable storage medium, which stores a computer program that can be loaded by a processor and execute the above editing and updating method. Through the description of the above implementation mode, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, including a number of instructions to enable a device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0282] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A virtual space interaction method based on AR technology, characterized in that: The following steps are involved: Obtaining viewing area information, wherein the viewing area is divided into a number of service areas, each of which has a diameter of less than 50 meters. Each service area corresponds to a base map reference plane containing location information, and each point on the base map reference plane corresponds to an editing operation area for placing materials; Acquire positioning information, and determine a working reference plane based on a correspondence between the positioning information and the service area, wherein the working reference plane is a base map reference plane corresponding to the positioning information; Determining the display orientation of the working reference surface based on the positioning information to determine the display surface; Acquire real-time on-site image information and match the working reference surface with the real-time on-site image information; Load the material into the editing area and adjust the material and playback timing based on preset requirements; Overlaying the display surface content of the visual operation area on the real-time on-site image information as output image information, wherein the visual operation area is the editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image; Displaying the superimposed image on a screen of a terminal device; The step of determining the display orientation of the working reference surface based on the positioning information includes: The display orientation of the working reference plane is adjusted based on the relative relationship between the reference orientation information and the real-time orientation information; wherein, the reference orientation information is the orientation information of the service area center coordinate point relative to the virtual coordinate point, the real-time orientation information is the orientation information of the terminal position coordinate point relative to the virtual coordinate point, and the virtual coordinate point is the reference point for establishing the base map reference plane.

2. The virtual space interaction method according to claim 1, characterized in that: The step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information includes: Acquire information of simulated positioning points within the service area, wherein each service area has multiple simulated positioning points; Obtain the simulated positioning point closest to the satellite positioning coordinate point and use it as the terminal position coordinate point; The preset orientation information corresponding to the terminal position coordinate point is read to determine the display orientation of the working reference surface, wherein the preset orientation information is pre-set information of the orientation of the base map reference surface with respect to the virtual coordinate point.

3. The virtual space interaction method according to claim 1, characterized in that: The editing operation area includes a two-dimensional plane with an orientation reference point, and the orientation reference point is located on the base map reference plane.

4. The virtual space interaction method according to claim 1, characterized in that: The editing operation area includes a three-dimensional space with an anchor reference point, and the anchor reference point is located on the base map reference plane.

5. The virtual space interaction method according to claim 1, characterized in that: The steps of obtaining positioning information and determining the working reference plane based on the correspondence between the positioning information and the service area include: Obtain real-time satellite positioning coordinate information and determine the service area based on the correlation between the distance difference between the satellite positioning coordinate point and the center coordinate point of each service area and the preset radius; where the center coordinate of the service area is the coordinate of the center of the service area, and the preset radius is the radius length of the service area; Obtain the basemap datum corresponding to the service area and load it as the working datum.

6. The virtual space interaction method according to claim 5, characterized in that: The steps of obtaining positioning information and determining the working reference plane based on the corresponding relationship between the positioning information and the service area also include: controlling the opening and closing of the camera module based on the relative position relationship between the satellite positioning coordinate point and the service area.

7. The virtual space interaction method according to claim 1, characterized in that: The step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information includes: Obtain real-time gyroscope information and determine whether the device enters or exits the raised state based on the real-time gyroscope information; Obtain the satellite positioning coordinate point corresponding to the moment the device enters the raised state and lock it as the terminal position coordinate point; Calculate the angle between the real-time orientation and the reference orientation, and adjust the displayed orientation of the working reference plane based on the angle.

8. The virtual space interaction method according to claim 7, characterized in that: The step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information further includes: Unlock the terminal position coordinate point when the device exits the raised state.

9. The virtual space interaction method according to claim 7, characterized in that: The step of adjusting the display orientation of the working reference surface based on the relative relationship between the reference orientation information and the real-time orientation information further includes: Calculate the distance difference between the real-time satellite positioning coordinates and the terminal position coordinates as the real-time drift distance; The terminal position coordinate point is updated based on the relative size of the real-time drift distance and the preset drift error distance, wherein the updated value of the terminal position coordinate point is the satellite positioning coordinate corresponding to when the real-time drift distance exceeds the preset drift error.

10. The virtual space interaction method according to claim 1, characterized in that: The steps of loading the material into the editing operation area and adjusting the material and the playback timing based on preset requirements include: Obtaining material placement information corresponding to the editing operation area on the work reference surface, and loading the material in the editing operation area based on the material placement information, wherein the material placement information is used to achieve preset requirements, including material file, material placement direction and / or material ratio information; Adjust the placement angle, color and / or size of the material in the editing operation area based on the material placement information; Set the playback timing of the material based on the timing file.

11. The virtual space interaction method according to claim 1, characterized in that: The steps of obtaining real-time on-site image information and matching the working reference surface with the real-time on-site image information include: Acquiring real-time on-site image information and real-time gyroscope information, wherein the real-time on-site image information and the real-time gyroscope information correspond to each other; Obtaining preset gyroscope information corresponding to the working reference surface, and rotating the working reference surface in real time based on the relative relationship between the preset gyroscope information and the real-time gyroscope information; The zoom of the material in the editing operation area on the working reference surface is controlled based on the camera module parameters.

12. The virtual space interaction method according to claim 1, characterized in that: The service areas are arranged at intervals and blank buffer zones are provided between adjacent service areas.

13. An editing and updating method, characterized in that: The method for interacting with a virtual space according to any one of claims 1 to 12 is applied to a server and comprises the following steps: Set the viewing area in the two-dimensional space of the map, and divide the viewing area into a service area and a blank buffer zone; Set the reference orientation based on the center coordinate point of the service area; Setting a number of virtual coordinate points in the three-dimensional map space, and setting a base map reference plane perpendicular to the reference orientation in the three-dimensional map space based on the virtual coordinate points, wherein the virtual coordinate points correspond to the service areas one by one; Several editing operation areas are set based on the base map reference plane, wherein the editing operation areas are used to load two-dimensional or three-dimensional materials; Select the material corresponding to the editing operation area, adjust the material based on the relative relationship between the editing operation area and the center coordinate point of the service area, and set the playback timing of the material; Push update information to terminal devices.

14. The editing and updating method according to claim 13, characterized in that: The method of adjusting the material includes stretching, scaling, rotating and / or translating the material.

15. The editing and updating method according to claim 13, wherein: The step of setting a plurality of editing operation areas based on the base map reference plane includes: An orientation reference point is set on the base map reference plane, and an editing operation area for placing two-dimensional materials is set based on the orientation reference point; An anchor reference point is set on the base map datum plane, and an editing operation area for placing three-dimensional materials is set based on the anchor reference point.

16. The editing and updating method according to claim 13, wherein: The editing operation area for placing two-dimensional materials is coplanar with or at an angle to the base map reference plane.

17. A virtual space interactive system based on AR technology, characterized in that: The virtual space interaction method according to any one of claims 1 to 12, comprising: A zone acquisition module is used to obtain viewing area information, wherein the viewing area is divided into a number of service areas, each of which has a diameter of less than 50 meters. Each service area corresponds to a base map reference plane containing location information, and each point on the base map reference plane corresponds to an editing operation area for placing materials; A positioning module, configured to obtain positioning information and determine a working reference plane based on a correspondence between the positioning information and the service area, wherein the working reference plane is a base map reference plane corresponding to the positioning information; The loading module is used to load materials into the editing operation area and adjust the materials and playback timing based on preset requirements; An orientation module, used to determine the display orientation of the working reference surface based on the positioning information; A matching module is used to obtain real-time on-site image information and match the working reference surface with the real-time on-site image information; A mapping module is used to superimpose the display surface content of the visual operation area on the real-time on-site image information as output image information, wherein the visual operation area is the editing operation area of ​​the working reference surface portion corresponding to the real-time on-site image; The display module is used to display the superimposed image on the screen of the terminal device.

18. An editing and updating system, characterized in that: The editing and updating method according to any one of claims 13 to 16 comprises: A zone division module is used to set a viewing area in the two-dimensional space of the map and divide the viewing area into a service area and a blank buffer zone, wherein the diameter of the service area is within 50 meters; A direction setting module is used to set a reference direction based on the center coordinate point of the service area; A reference setting module is used to set a number of virtual coordinate points in the three-dimensional map space, and a base map reference plane perpendicular to the reference direction is set in the three-dimensional map space based on the virtual coordinate points, wherein the virtual coordinate points correspond to the service areas one by one; An editing area setting module is used to set a number of editing operation areas based on the base map reference plane, wherein the editing operation area is used to load two-dimensional or three-dimensional materials; The editing module is used to select the material corresponding to the editing operation area and adjust the material based on the relative relationship between the editing operation area and the center coordinate point of the service area; The push module is used to push update information to terminal devices.

19. An electronic device, characterized in that: It includes: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to: Execute the virtual space interaction method based on AR technology according to any one of claims 1 to 12.

20. A server, characterized in that: It includes: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to: Execute the editing and updating method according to any one of claims 13 to 16.

21. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement: The virtual space interaction method based on AR technology according to any one of claims 1 to 12; The editing and updating method according to any one of claims 13 to 16.

Images