Pipeline network management method based on virtual reality and augmented reality

Abstract

The present invention discloses a pipeline network management method based on virtual reality and augmented reality, belonging to the field of image communication technology. The method comprises: establishing a virtual map and a three-dimensional pipeline model, dividing the virtual map into a plurality of first areas, and dividing the three-dimensional pipeline model into a plurality of second areas; sending a transmission request to a server based on a handheld terminal, the transmission request comprising a first request and a second request; if the server receives the first request, obtaining the corresponding first area and the second area, merging the first area and the second area into a VR image and sending it to the handheld terminal; if the server receives the second request, sending the three-dimensional pipeline model in the corresponding first area as an AR image to the handheld terminal; if it is a VR image, the handheld terminal directly displays it, and if it is an AR image, the handheld terminal displays it in a real scene image. The present invention can send only VR and AR images of part of the pipeline area according to demand, thereby greatly improving the image sending speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image transmission, and in particular relates to a pipe network management method based on virtual reality and augmented reality. Background Art

[0002] Underground pipelines are responsible for important functions such as energy dispatching, information transmission, and drainage and flood control. However, with the development of cities, problems such as improper pipeline locations and chaotic and overlapping geographical spaces have emerged. As a result, accidents of pipelines being cut during construction have occurred frequently. Therefore, pipeline detection, inspection, repair, and system visualization management are necessary to ensure the safe operation of major nuclear-related plants and urban underground pipelines to ensure the safe operation of nuclear-related plants and protect urban underground space.

[0003] To effectively solve the management problem of underground pipelines, the following methods are proposed in the prior art. For example, Chinese patent document 1 with publication number CN115511113A discloses a pipeline management system and management method. The system includes multiple modules, among which the pipeline visualization management platform renders the scene pipelines of the entire area in 3D, which is used for users to determine the location of the pipelines in the scene and understand the specific conditions of the pipelines. The AR field inspection platform uses AR technology to display the pipeline distribution of the area on mobile devices. This method realizes the discovery of problems on the remote management platform, and facilitates the verification of the discovered problems on the spot through AR technology, thereby improving the efficiency of pipeline maintenance. For another example, Chinese patent document 2 with publication number CN109246195A discloses a pipeline network intelligent management and control method and system integrating augmented reality and virtual reality. This method superimposes the ground real scene with the virtual model of the underground pipeline network and the real-time monitoring data of the Internet of Things. After the sensor data is transmitted to the augmented reality terminal, the sensor information is superimposed on the sensor three-dimensional model and the real scene image through the data display module of the augmented reality system unit, thereby improving the efficiency of pipeline network inspection.

[0004] Both of the above methods first establish a three-dimensional model of the regional underground pipeline, and then send the three-dimensional model to the user's display terminal according to the request. Different from traditional image data, VR and AR images require more network bandwidth when sending due to their large data volume. In addition, the user's display terminal generally receives data through a wireless network. If the network of the display terminal is not good, there will be a large delay in obtaining VR and AR images, which will affect the efficiency of pipeline inspection. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a pipe network management method based on virtual reality and augmented reality to solve the problems existing in the above background technology.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes a pipe network management method based on virtual reality and augmented reality, comprising:

[0007] In the server, a virtual map is established based on the city map, a three-dimensional pipeline model is established based on the pipeline design drawing, and the three-dimensional pipeline model is matched with the virtual map;

[0008] Dividing the matched virtual map into a plurality of first areas, and dividing the three-dimensional pipeline model into a plurality of second areas, wherein the first areas correspond to the second areas one by one;

[0009] Sending a transmission request to a server based on a handheld terminal, the transmission request includes a first request and a second request, the first request is used to obtain a VR image, and the second request is used to obtain an AR image;

[0010] If the server receives the first request, it obtains the first area and the second area corresponding to the request position specified in the first request, merges the first area and the second area into the VR image, and sends the VR image together with the corresponding pipeline parameters to the handheld terminal;

[0011] If the server receives the second request, it obtains the corresponding first area according to the positioning position in the second request, uses the three-dimensional pipeline model in the first area as the AR image, and sends it together with the corresponding pipeline parameters to the handheld terminal;

[0012] If it is the VR image, the handheld terminal directly displays it; if it is the AR image, the handheld terminal superimposes the AR image on the real scene image for display.

[0013] Furthermore, the server sending the VR image includes the following steps:

[0014] The first area and the second area are merged into a virtual space, wherein an initial viewpoint and an initial viewing angle are set in the virtual space, and the server intercepts the VR image in the virtual space within a preset range, and the VR image includes a central image and an edge image, wherein the central image is a field of view image corresponding to the initial viewpoint and the initial viewing angle in the virtual space, and the edge image is a portion of the VR image other than the central image, and the central image is set to a first resolution and the edge image is set to a second resolution;

[0015] After receiving the VR image, the handheld terminal detects user operations, which include viewpoint movement and viewing angle movement. When the user operation meets the preset conditions, the handheld terminal returns the user viewpoint and user viewing angle when the preset conditions are triggered to the server. The server captures the VR image again based on the user viewpoint and user viewing angle and returns it to the handheld terminal.

[0016] Further, after returning the VR image multiple times, the server adjusts the preset range based on the following steps:

[0017] It is defined that the user operation received by the handheld terminal between the i-th VR image and the i+1-th VR image is an operation record corresponding to the i-th VR image, and the handheld terminal returns the operation record to the server. After collecting the operation records of N VR images, the server calculates the evaluation value of each of the N VR images based on a first formula, and the first formula is: , wherein Di is the evaluation value of the i-th VR image among the N VR images, Vi is the maximum viewing angle rotation speed appearing in the operation record of the i-th VR image, Mi is the maximum viewing angle rotation angle, Fi is the maximum viewpoint movement amount, ai is the maximum viewing angle movement acceleration, ω1, ω2, ω3 and ω4 are the first weight, the second weight, the third weight and the fourth weight;

[0018] An average value of the evaluation values ​​of the N VR images is calculated, and a standard value is set. If the average value is greater than the standard value, the preset range is increased; if the average value is less than the standard value, the preset range is reduced.

[0019] Furthermore, a trigger area is marked in the virtual space, and when the user's viewpoint moves to the trigger area, it is determined that the user operation meets the preset condition.

[0020] Furthermore, the handheld terminal locates the projection point of the visual center line in the AR image based on the user's perspective; if the projection point is located in the central image and the amount of movement within a preset time period is less than a first threshold, the AR image at the location of the projection point is enlarged at a first ratio with the projection point as the center; if the amount of movement of the projection point is still less than the first threshold after enlargement and after the preset time period has passed again, the AR image is further enlarged at a second ratio, and the second ratio is smaller than the first ratio.

[0021] Further, the edge image of the second resolution is updated to the first resolution based on the following steps:

[0022] Divide the edge image into a plurality of sub-regions, obtain historical data, assign a number to each of the sub-regions in the historical data, define the sub-region corresponding to the initial viewpoint and the initial viewing angle as a starting region, the number is the order in which the projection point passes through each of the sub-regions starting from the starting region in the historical data, and the number of the starting region is 0;

[0023] Based on the historical data, the number of times each sub-region is assigned the number with different values ​​is counted, and the sub-region that is assigned the number 1 and the most times is defined as a sending order 1, and the sub-region that is assigned the number 2 and the most times is defined as a sending order 2, and this step is repeated until a sending order with a corresponding value is assigned to each sub-region;

[0024] After sending the current VR image, the sub-regions of the first resolution are sent in sequence based on a sending order to update the edge image of the second resolution to the first resolution.

[0025] Furthermore, sending the AR image includes the following steps:

[0026] The server obtains the network status between the server and the handheld terminal, determines the animation frame rate based on the network status, converts the AR image into a dynamic image according to the determined animation frame rate, and sends the dynamic image to the handheld terminal.

[0027] Furthermore, matching the three-dimensional pipeline model with the real scene map comprises the following steps:

[0028] A plurality of first reference points are selected in the three-dimensional pipeline model, geographic coordinates of the first reference points are obtained, second reference points are located in the real-life map based on the geographic coordinates, and the three-dimensional pipeline model is matched with the real-life map based on the first reference points and the second reference points.

[0029] Furthermore, if the handheld terminal changes the pipeline parameters, the user information and the changed pipeline parameters are recorded, and the user information includes the user name and the change time. The handheld terminal returns the user information and the changed pipeline parameters to the server, and the server updates the pipeline parameters and records the user information.

[0030] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0031] The present invention first matches the virtual map with the three-dimensional pipeline model, then divides the virtual map into a first area, and divides the three-dimensional pipeline model into a second area, thereby dividing the virtual map and the three-dimensional pipeline model into multiple small areas; after receiving the transmission request of the handheld terminal, the server module first determines the type of the transmission request, if it is a first request, then sends a VR image of the corresponding small area according to the position requested by the user, if it is a second request, then only sends an AR image of the small area corresponding to the user's own position. Since the server module only sends a partial area in the entire image, the transmission amount of the image data is reduced, so that even when the network condition of the handheld terminal is poor, the image acquisition delay can be greatly reduced; in addition, the user can place himself in the virtual world in the handheld terminal to browse the pipeline distribution, and can also view the distribution of underground pipelines in the real world, thereby greatly improving the convenience of pipeline management. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of the steps of the pipe network management method based on virtual reality and augmented reality of the present invention;

[0033] Figure 2 This is a schematic diagram of the VR image principle generated by the present invention;

[0034] Figure 3 A schematic diagram of the principle of dividing an edge image into sub-regions according to the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0037] like Figure 1 As shown, a pipe network management method based on virtual reality and augmented reality includes:

[0038] S1: In the server, a virtual map is established based on the city map, a three-dimensional pipeline model is established based on the pipeline design drawings, and the three-dimensional pipeline model is matched with the virtual map.

[0039] S2: Divide the matched virtual map into a plurality of first areas, and divide the three-dimensional pipeline model into a plurality of second areas, wherein the first areas correspond to the second areas one by one.

[0040] The present invention first establishes a virtual map that highly restores the city map scene, and establishes a three-dimensional pipeline model in a three-dimensional space that is the same as the actual pipeline distribution according to the distribution of pipelines in the pipeline design drawings, and then places the three-dimensional pipeline model in the virtual map, so that the two can be integrated. Relevant personnel can know the underground of which buildings the pipelines are specifically distributed when viewing the integrated map, and can obtain a more intuitive distribution of urban pipelines, which can be reasonably planned before construction work is carried out. After the virtual map is matched with the three-dimensional pipeline model, the virtual map and the three-dimensional pipeline model are divided in the form of a grid to obtain multiple first areas and second areas. Since the matching has been performed before, the three-dimensional pipeline model included in the second area is distributed under the first area, so there will be a one-to-one correspondence between the first area and the second area.

[0041] S3: Sending a transmission request to the server based on the handheld terminal, where the transmission request includes a first request and a second request, where the first request is used to obtain a VR image, and the second request is used to obtain an AR image.

[0042] The handheld terminal is specifically a mobile terminal such as a mobile phone with display function and wireless communication function. The relevant personnel send a transmission request for transmitting image data to the server through the handheld terminal, and the transmission request includes a first request and a second request. The following is a brief introduction to VR images and AR images. VR images build a virtual world, and users place themselves in the virtual world for browsing. AR images superimpose the virtual world on the real world for display. When a user needs to browse the distribution of underground pipelines in a certain area in the office, he needs to use a handheld terminal to obtain a VR image. When a user is at a city site and needs to determine the distribution of underground pipelines at his location, he needs to obtain an AR image.

[0043] S4: If the server receives the first request, it obtains the corresponding first area and second area according to the request position specified in the first request, merges the first area and the second area into a VR image, and sends it to the handheld terminal together with the corresponding pipeline parameters.

[0044] S5: If the server receives the second request, it obtains the corresponding first area according to the positioning position in the second request, takes the three-dimensional pipeline model in the first area as an AR image, and sends it to the handheld terminal together with the corresponding pipeline parameters.

[0045] S6: If it is a VR image, the handheld terminal directly displays it; if it is an AR image, the handheld terminal superimposes the AR image on the real scene image for display.

[0046] When a user needs to obtain a VR image, he needs to specify his position in the virtual world in advance, that is, the requested position, and then merge the first area and the second area including the requested position into a VR image for sending. When the user needs an AR image, since the user himself is in the actual site, only the three-dimensional pipeline model of the corresponding position is sent. In addition, when sending VR images and AR images, the server also sends pipeline parameters such as water flow, burial depth, etc., for user viewing.

[0047] By splitting the virtual map and pipeline 3D model into multiple areas, only the required area can be sent according to the requested location instead of sending the entire virtual map and 3D pipeline model. This greatly reduces the transmission volume of image data and reduces transmission delay.

[0048] The present invention first matches the virtual map with the three-dimensional pipeline model, then divides the virtual map into a first area, and divides the three-dimensional pipeline model into a second area, thereby dividing the virtual map and the three-dimensional pipeline model into multiple small areas; after receiving the transmission request of the handheld terminal, the server module first determines the type of the transmission request, if it is a first request, then sends a VR image of the corresponding small area according to the position requested by the user, if it is a second request, then only sends an AR image of the small area corresponding to the user's own position. Since the server module only sends a partial area in the entire image, the transmission amount of the image data is reduced, so that even when the network condition of the handheld terminal is poor, the image acquisition delay can be greatly reduced; in addition, the user can place himself in the virtual world in the handheld terminal to browse the pipeline distribution, and can also view the distribution of underground pipelines in the real world, thereby greatly improving the convenience of pipeline management.

[0049] It is particularly noteworthy that the present invention can send VR and AR images of only part of the pipeline area according to demand, thereby greatly improving the image sending speed.

[0050] In this embodiment, the server sends the VR image including the following steps:

[0051] The first area and the second area are merged into a virtual space, in which an initial viewpoint and an initial viewing angle are set. The server captures a VR image in the virtual space within a preset range, and the VR image includes a central image and an edge image. The central image is a field of view image corresponding to the initial viewpoint and the initial viewing angle in the virtual space, and the edge image is the portion of the VR image other than the central image. The central image is set to a first resolution, and the edge image is set to a second resolution.

[0052] After receiving the VR image, the handheld terminal detects the user operation, which includes viewpoint movement and viewing angle movement. When the user operation meets the preset conditions, the handheld terminal returns the user viewpoint and user viewing angle when the preset conditions are triggered to the server. The server captures the VR image again based on the user viewpoint and user viewing angle and returns it to the handheld terminal.

[0053] Specifically, according to the following steps, it is determined whether the preset condition is triggered. A trigger area is marked in the virtual space. When the user's viewpoint moves to the trigger area, it is determined that the user operation meets the preset condition.

[0054] The first area and the second area are superimposed into a virtual space, and then a virtual camera is set in the virtual space. The camera has an initial viewpoint and an initial viewing angle, which is equivalent to the initial viewpoint and initial viewing angle of the user. The initial viewpoint is the position of the user, expressed in coordinates, and the initial viewing angle is the size of the area covered by the user's gaze, expressed in angles. In particular, the request position is only used to determine which area to send, and does not conflict with the initial viewpoint in this step.

[0055] After the virtual space is generated, the VR image is intercepted within a preset range. The preset range is specifically the radius of the circle, which is defined as the interception radius. When intercepting, the initial viewpoint P1 is used as the center of the circle and the interception radius is used for surrounding shooting. Then, the captured images are spliced ​​into a spherical surrounding image P3. Figure 2 P2 is the area covered by the initial viewing angle, the image corresponding to the covered area is the central image, and the images other than the central image in the surround image P3 are edge images. Since the central image is the first image viewed by the user, the central image is sent with a higher first resolution, and the edge images are sent with a lower second resolution. Subsequently, at intervals, the edge images are changed to the first resolution. Through this step, the amount of data required to send the image can be further reduced during the first transmission. In the subsequent process of the user viewing the central image, the resolution of the edge images can be gradually improved to avoid affecting the user's viewing experience.

[0056] When the handheld terminal displays the VR image, it also continuously detects the user operation. When the user operation triggers the preset condition, the handheld terminal automatically requests the server to obtain the VR image of the corresponding viewpoint again, such as setting a detection area near the edge of the virtual space. Figure 2 In area P5, when the user moves from position P2 to position P4 and falls into the detection area, the server will generate a new VR image again within the preset range with the user's viewpoint P4 as the center and return it to the handheld terminal.

[0057] After returning VR images multiple times, the server of this embodiment adjusts the preset range based on the following steps:

[0058] It is defined that the user operation of the handheld terminal between receiving the i-th VR image and the i+1-th VR image is the operation record corresponding to the i-th VR image. The handheld terminal returns the operation record to the server. After collecting the operation records of N VR images, the server calculates the evaluation value of each of the N VR images based on the first formula. The first formula is: , where Di is the evaluation value of the i-th VR image in N VR images, Vi is the maximum viewing angle rotation speed that appears in the i-th VR image operation record, Mi is the maximum viewing angle rotation, Fi is the maximum viewpoint movement, ai is the maximum viewing angle movement acceleration, ω1, ω2, ω3 and ω4 are the first weight, the second weight, the third weight and the fourth weight.

[0059] Calculate the average of N VR image evaluation values, set a standard value, and if the average value is greater than the standard value, increase the preset range; if the average value is less than the standard value, reduce the preset range.

[0060] After the handheld terminal receives the i-th VR image, it may perform operations of viewpoint movement and visual angle movement. These operations will be recorded until the handheld terminal receives the i+1-th VR image. These operations will be considered as operations corresponding to the i-th VR image. The display module will extract the maximum visual angle rotation speed, maximum visual angle rotation angle, maximum viewpoint movement speed and maximum visual angle movement acceleration from these operations. The visual angle rotation speed is the speed of change of the user's visual angle in a single rotation, which is obtained by dividing the visual angle of rotation by the duration of the rotation process. The visual angle of rotation is the angle of change of the user's visual angle in a single rotation. The viewpoint movement amount is the distance that the user's viewpoint moves in a single movement. The visual angle movement acceleration is calculated based on the visual angle movement speed and movement time.

[0061] The larger the evaluation value calculated in the first formula, the greater the degree of change in the user's line of sight in the VR image, which means that the preset range may be set too small and the image is too close to the user's field of view, so the user needs to frequently move and change the viewpoint to view, which increases the user's operating burden. If the average value is greater than the standard value, it means that multiple VR images have the above problems. At this time, it is necessary to increase the preset range to reduce the user's operating burden. Otherwise, it means that the preset range is set too large, which may cause the user to be unable to see the VR image clearly and need to reduce the preset range. The standard value can be determined by testing personnel and based on the test experience results.

[0062] The handheld terminal locates the projection point of the visual center line in the AR image based on the user's perspective. If the projection point is located in the central image and the movement within a preset time period is less than a first threshold, the AR image at the location of the projection point is enlarged at a first ratio with the projection point as the center. If the movement of the projection point is still less than the first threshold after enlargement and a preset time period has passed again, the AR image is further enlarged at a second ratio, and the second ratio is smaller than the first ratio.

[0063] Continue to refer to Figure 2 , when the user's viewpoint is P1 and the user's viewing angle is P2, the projection point of the visual centerline in the AR image is P6. If the movement of the projection point within the preset time is less than the first threshold, the preset time is, for example, 5s, and the first threshold is, for example, 1m, then it is considered that the user is looking at the location of the projection point. In order to help the user better observe the place, the system automatically zooms in on the image at the projection point. If it is detected that the user is still staring at the point after zooming in, the place is zoomed in again, and the second zoom ratio is smaller than the first zoom ratio of the last zoom, to avoid the image being too close to the user's line of sight and affecting viewing; in particular, the user can manually return to the original zoom ratio after viewing. Through this step, the image can be automatically zoomed in according to the user's viewing situation, thereby reducing user operations and improving user experience.

[0064] This embodiment updates the edge image of the second resolution to the first resolution based on the following steps:

[0065] The edge image is divided into multiple sub-regions, historical data is obtained, and each sub-region in the historical data is numbered. The sub-region corresponding to the initial viewpoint and the initial viewing angle is defined as the starting region. The numbering is the order in which the projection point passes through each sub-region with the starting region as the starting point in the historical data. The starting region is numbered 0.

[0066] like Figure 3 As shown, A0 is the center image, and the rest is the edge image. The edge image is divided into multiple rectangular sub-areas A1-A8, and then historical data is obtained. In each piece of historical data, since the initial viewpoint and initial viewing angle are set in advance, the projection point of the visual center line will be fixed in the sub-area A0. Here, the sub-area number is set to 0. When the user's viewpoint and viewing angle begin to change, the corresponding projection point will also move. If the projection point moves from sub-area A0 to sub-area A2, then the sub-area A2 is numbered 1. The same applies to the numbering of sub-areas in other historical data.

[0067] Based on historical data, count the number of times each sub-area is assigned different numerical numbers, define the sub-area assigned number 1 and the most times as sending order 1, and define the sub-area assigned number 2 and the most times as sending order 2. Repeat this step until each sub-area is assigned a sending order with a corresponding numerical value.

[0068] After sending the current VR image, the sub-regions of the first resolution are sent in sequence based on the sending order to update the edge image of the second resolution to the first resolution.

[0069] Assume that there are 100 historical data in the database, among which sub-region A2 is assigned the number 2 90 times, and the number of times the remaining sub-regions are assigned the number 2 is less than 90 times, so sub-region A2 is defined as sending order 1. Similarly, assuming that the subsequent sub-region A4 is defined as sending order 2, and sub-region A3 is defined as sending order 3, after the server sends the VR image, it first sends the image corresponding to sub-region A2 of the first resolution again. After the handheld terminal receives it, it uses it to replace the sub-region A2 corresponding to the second resolution, thereby completing the update of sub-region A2 from the second resolution to the first resolution. After that, the server sends sub-region A4 again, and then sends sub-region A3. Through this step, the resolution of the edge image can be sequentially improved according to the user's browsing habits, thereby greatly reducing the user's discomfort during browsing on the basis of improving the image sending speed.

[0070] In this embodiment, sending an AR image includes the following steps:

[0071] The server obtains the network status between the handheld terminal and the server, determines the animation frame rate based on the network status, converts the AR image into a dynamic image according to the determined animation frame rate, and sends the dynamic image to the handheld terminal.

[0072] The network status is specifically the data packet loss rate. When the data packet loss rate is less than the second threshold, the network status is considered to be good, and the AR image is generated and sent at a higher first frame rate. When the data packet loss rate is greater than or equal to the second threshold, the network status is considered to be poor, and the AR image is generated and sent at a lower second frame rate. By superimposing dynamic images on real-scene images, the user experience can be further improved, and by adjusting the frame rate according to the network status, the user's reception delay can be reduced.

[0073] In this embodiment, matching the three-dimensional pipeline model with the real-scene map includes the following steps:

[0074] A plurality of first reference points are selected in the three-dimensional pipeline model, the geographic coordinates of the first reference points are obtained, the second reference points are located in the real-life map based on the geographic coordinates, and the three-dimensional pipeline model is matched with the real-life map based on the first reference points and the second reference points.

[0075] The geographic coordinates are specifically longitude and latitude coordinates. By using the first reference point and the second reference point with the same geographic coordinates to align the three-dimensional pipeline model with the real-life map, the matching efficiency between the two can be greatly improved.

[0076] If the handheld terminal changes the pipeline parameters, the user information and the changed pipeline parameters are recorded. The user information includes the user name and the change time. The handheld terminal returns the user information and the changed pipeline parameters to the server, and the server updates the pipeline parameters and records the user information.

[0077] In this embodiment, the pipeline parameters can be changed through a handheld terminal. When the change is made, the user information performing the change operation will be recorded, and the changed pipeline parameters and user information will be uploaded to the server. The server will record the pipeline parameters before and after the change, and will also record which users made the changes and when, thereby further improving the pipeline management level.

[0078] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0079] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A pipe network management method based on virtual reality and augmented reality, characterized in that: include: In the server, a virtual map is established based on the city map, a three-dimensional pipeline model is established based on the pipeline design drawing, and the three-dimensional pipeline model is matched with the virtual map; Dividing the matched virtual map into a plurality of first areas, and dividing the three-dimensional pipeline model into a plurality of second areas, wherein the first areas correspond to the second areas one by one; Sending a transmission request to a server based on a handheld terminal, the transmission request includes a first request and a second request, the first request is used to obtain a VR image, and the second request is used to obtain an AR image; If the server receives the first request, it obtains the first area and the second area corresponding to the request position specified in the first request, merges the first area and the second area into the VR image, and sends the VR image together with the corresponding pipeline parameters to the handheld terminal; If the server receives the second request, it obtains the corresponding first area according to the positioning position in the second request, uses the three-dimensional pipeline model in the first area as the AR image, and sends it together with the corresponding pipeline parameters to the handheld terminal; If it is the VR image, the handheld terminal directly displays it; if it is the AR image, the handheld terminal superimposes the AR image on the real scene image for display; The server sending the VR image includes the following steps: The first area and the second area are merged into a virtual space, wherein an initial viewpoint and an initial viewing angle are set in the virtual space, and the server intercepts the VR image in the virtual space within a preset range, and the VR image includes a central image and an edge image, wherein the central image is a field of view image corresponding to the initial viewpoint and the initial viewing angle in the virtual space, and the edge image is a portion of the VR image other than the central image, and the central image is set to a first resolution and the edge image is set to a second resolution; After receiving the VR image, the handheld terminal detects the user operation, which includes viewpoint movement and visual angle movement. When the user operation meets the preset condition, the handheld terminal returns the user viewpoint and user visual angle when the preset condition is triggered to the server, and the server captures the VR image again according to the user viewpoint and user visual angle and returns it to the handheld terminal; After returning the VR image multiple times, the server adjusts the preset range based on the following steps: It is defined that the user operation received by the handheld terminal between the i-th VR image and the i+1-th VR image is an operation record corresponding to the i-th VR image, and the handheld terminal returns the operation record to the server. After collecting the operation records of N VR images, the server calculates the evaluation value of each of the N VR images based on the first formula, and the first formula is: D i =V i ω1+M i ω2+F i ω3+a i ω4, where D i is the evaluation value of the i-th VR image among the N VR images, V i is the maximum viewing angle rotation speed in the operation record of the i-th VR image, M i is the maximum viewing angle, F i is the maximum viewpoint movement, a i is the maximum viewing angle movement acceleration, ω1, ω2, ω3 and ω4 are the first weight, the second weight, the third weight and the fourth weight; An average value of the evaluation values ​​of the N VR images is calculated, and a standard value is set. If the average value is greater than the standard value, the preset range is increased; if the average value is less than the standard value, the preset range is reduced.

2. The method according to claim 1, characterized in that A trigger area is marked in the virtual space, and when the user's viewpoint moves to the trigger area, it is determined that the user operation meets the preset condition.

3. The method according to claim 1, characterized in that The handheld terminal locates the projection point of the visual center line in the AR image based on the user's perspective. If the projection point is located in the central image and the movement amount within a preset time period is less than a first threshold, the AR image at the location of the projection point is enlarged at a first ratio with the projection point as the center. If the movement amount of the projection point is still less than the first threshold after enlargement and the preset time period has passed again, the AR image is further enlarged at a second ratio, and the second ratio is smaller than the first ratio.

4. The method according to claim 3, characterized in that The edge image of the second resolution is updated to the first resolution based on the following steps: Divide the edge image into a plurality of sub-regions, obtain historical data, assign a number to each of the sub-regions in the historical data, define the sub-region corresponding to the initial viewpoint and the initial viewing angle as a starting region, the number is the order in which the projection point passes through each of the sub-regions starting from the starting region in the historical data, and the number of the starting region is 0; Based on the historical data, the number of times each sub-region is assigned the number with different values ​​is counted, and the sub-region that is assigned the number 1 and the most times is defined as a sending order 1, and the sub-region that is assigned the number 2 and the most times is defined as a sending order 2, and this step is repeated until a sending order with a corresponding value is assigned to each sub-region; After sending the current VR image, the sub-regions of the first resolution are sent in sequence based on a sending order to update the edge image of the second resolution to the first resolution.

5. The method according to claim 1, characterized in that Sending the AR image includes the following steps: The server obtains the network status between the server and the handheld terminal, determines the animation frame rate based on the network status, converts the AR image into a dynamic image according to the determined animation frame rate, and sends the dynamic image to the handheld terminal.

6. The method according to claim 1, characterized in that Matching the three-dimensional pipeline model with the virtual map comprises the following steps: A plurality of first reference points are selected in the three-dimensional pipeline model, geographical coordinates of the first reference points are obtained, second reference points are located in the virtual map based on the geographical coordinates, and the three-dimensional pipeline model is matched with the virtual map based on the first reference points and the second reference points.

7. The method according to claim 1, characterized in that If the handheld terminal changes the pipeline parameters, the user information and the changed pipeline parameters are recorded, and the user information includes the user name and the change time. The handheld terminal returns the user information and the changed pipeline parameters to the server, and the server updates the pipeline parameters and records the user information.

Images