Business service method and system based on digital twin

By using digital twin technology in intelligent business services to build and update three-dimensional models and environmental information, combined with sensor data, the problems of high sensor costs and limited detection range are solved, and more efficient business operation decisions and autonomous driving speed are achieved.

CN118714186BActive Publication Date: 2024-11-15LITUREX GUANGZHOU CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411194908.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-15
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The existing intelligent business services cannot effectively utilize the three-dimensional data of application scenarios, resulting in high cost of sensor application and maintenance, and fields such as autonomous driving are limited by the detection range of sensors and their speeds are limited.

Method used

Using a business service method and system based on digital twins, the initial digital twin model is built through a three-dimensional world server. The business client receives and updates the three-dimensional model and environmental information, and corrects it in combination with sensor data to generate target three-dimensional model and environmental information for execution. Business operations.

Benefits of technology

It reduces the number of sensors required by the business client, reduces costs, improves the vision and driving speed of intelligent logistics and autonomous driving, and realizes business operation decisions on real-time three-dimensional models and environmental information of the target area.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118714186B_ABST
    Figure CN118714186B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of digital twin technology, and specifically discloses a business service method and system based on digital twins, wherein a three-dimensional world server constructs an initial digital twin model of a target area, a business client sends a business service request for the target area to the three-dimensional world server, and the three-dimensional world server sends the initial three-dimensional model and initial environmental information of the target area to the business client, the business client receives sensor data collected by sensors, and reconstructs the initial three-dimensional model according to the sensor data and updates the initial environmental information to obtain a target three-dimensional model and target environmental information, and the business client performs business operations according to the target three-dimensional model and target environmental information. Business operation decisions are made through a real-time digital twin three-dimensional model of the target area including a three-dimensional model and environmental information, which can reduce the sensors required by the business client, reduce costs, and avoid business services being restricted by some sensors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of digital twins, and in particular, relates to a business service method and system based on digital twins. Background Art

[0002] With the development of artificial intelligence, intelligent unmanned technologies have emerged in various businesses, such as intelligent inspection, intelligent logistics, autonomous driving, disaster emergency warning and other intelligent businesses.

[0003] In the above-mentioned intelligent services, such as the intelligent bridge inspection service, the existing technology is to install monitoring sensors such as stress sensors, strain gauges, accelerometers, etc. on the lower bridge. It does not use the three-dimensional data of the bridge and its surrounding environment, requires the support of various sensors, and has high maintenance and application costs. Or in the autonomous driving scenario, after planning the driving route through GPS positioning, the path is corrected by real-time detection such as on-board images and radars. The autonomous driving speed is limited by the detection range of the sensor, which limits the speed of autonomous driving.

[0004] Existing intelligent business services are unable to use the three-dimensional data of application scenarios, use too many sensors, resulting in high sensor application and maintenance costs, and are limited by sensor performance. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a business service method and system based on digital twins, aiming to solve the problem that three-dimensional data of application scenarios cannot be used in business services.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] A business service method based on digital twins is applied to a business service system including a three-dimensional world server and a business client, and specifically includes the following steps:

[0008] The three-dimensional world server constructs an initial digital twin model of the target area, wherein the initial digital twin model includes an initial three-dimensional model and initial environmental information of the target area;

[0009] The business client sends a business service request for the target area to the three-dimensional world server;

[0010] After receiving the business service request, the 3D world server sends the initial 3D model and initial environment information of the target area to the business client;

[0011] The service client receives the initial three-dimensional model and initial environment information;

[0012] The service client receives the sensor data collected by the sensor, and reconstructs the initial three-dimensional model and updates the initial environment information according to the sensor data to obtain the target three-dimensional model and target environment information of the target area;

[0013] The business client performs business operations according to the target three-dimensional model and the target environment information.

[0014] As a further technical solution of the present invention, the following steps are also included:

[0015] The business client uploads the target three-dimensional model and target environment information as a target digital twin model of the target area to the three-dimensional world server;

[0016] The three-dimensional world server uses the target digital twin model to replace the initial digital twin model as the updated initial digital twin model.

[0017] As a further technical solution of the present invention, the service client receives the sensor data collected by the sensor, and reconstructs the initial three-dimensional model and updates the initial environment information according to the sensor data to obtain the target three-dimensional model and target environment information of the target area, which specifically includes the following steps:

[0018] The service client receives current environmental information collected by the environmental sensor, and receives surveying and mapping data collected by multiple surveying and mapping sensors;

[0019] Preprocessing the current environmental information, and determining the preprocessed environmental information as the target environmental information of the target area;

[0020] Preprocessing the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data;

[0021] Fusing the preprocessed surveying and mapping data to obtain fused data;

[0022] The initial three-dimensional model is modified using the fusion data to obtain a target three-dimensional model of the target area.

[0023] As a further technical solution of the present invention, preprocessing the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data specifically includes the following steps:

[0024] The surveying and mapping data collected by each surveying and mapping sensor are deduplicated, deduplicated outliers are removed, and filtered to obtain preprocessed surveying and mapping data.

[0025] As a further technical solution of the present invention, the surveying and mapping data includes multiple images and multiple first point clouds, and the pre-processed surveying and mapping data are fused to obtain fused data, which specifically includes the following steps:

[0026] Generate a second point cloud according to the preprocessed image, and fuse the first point cloud and the second point cloud to obtain a fused point cloud;

[0027] Multiple images are stitched together to obtain a stitched image.

[0028] As a further technical solution of the present invention, the initial three-dimensional model is corrected by using the fusion data to obtain a target three-dimensional model of the target area, which specifically includes the following steps:

[0029] Inputting the fused point cloud and the initial three-dimensional model into a physical object recognition model to obtain a differentiated entity;

[0030] Correcting the mesh of the differentiated entity in the initial three-dimensional model according to the point cloud of the differentiated entity in the fused point cloud to obtain a corrected three-dimensional model;

[0031] The spliced ​​image is used as a texture map to perform texture rendering on the corrected three-dimensional model to obtain a target three-dimensional model of the target area.

[0032] As a further technical solution of the present invention, the business client performs a business operation according to the target three-dimensional model and the target environment information, including:

[0033] Loading initial business operation information, where the initial business operation information is used to instruct the business client to perform a business operation;

[0034] Modifying the initial business operation information according to the target three-dimensional model, the target environment information and a preset business algorithm to obtain target business operation information;

[0035] Execute a business operation according to the target business operation information.

[0036] As a further technical solution of the present invention, the business service includes one of intelligent inspection business service, intelligent logistics distribution business service, assisted driving business service, disaster emergency and warning business server, and virtual reality business service.

[0037] A business service system based on digital twins, specifically comprising a three-dimensional world server and a business client, wherein the three-dimensional world server comprises a digital twin model maintenance unit and a business service request response unit, and the business client comprises a business request unit, an initial digital twin model receiving unit, a digital twin model reconstruction unit, and a business execution unit;

[0038] The digital twin model maintenance unit in the three-dimensional world server is used to construct an initial digital twin model of the target area, wherein the initial digital twin model includes an initial three-dimensional model and initial environmental information of the target area;

[0039] The service request unit in the service client is used to send a service request for the target area to the three-dimensional world server;

[0040] The business service request response unit in the three-dimensional world server is used to send the initial three-dimensional model and initial environment information of the target area to the business client after receiving the business service request;

[0041] The initial digital twin model receiving unit in the business client is used to receive the initial three-dimensional model and initial environment information;

[0042] The digital twin model reconstruction unit in the business client is used to receive the sensor data collected by the sensor, and reconstruct the initial three-dimensional model and update the initial environmental information according to the sensor data to obtain the target three-dimensional model and target environmental information of the target area;

[0043] The service execution unit in the service client is used to execute service operations according to the target three-dimensional model and the target environment information.

[0044] As a further technical solution of the present invention, the service client further includes:

[0045] A model uploading unit, configured to upload the target three-dimensional model and target environment information as a target digital twin model of the target area to the three-dimensional world server;

[0046] The three-dimensional world server also includes:

[0047] A digital twin model updating unit is used to replace the initial digital twin model with the target digital twin model as the updated initial digital twin model.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention is based on digital twin technology. An initial digital twin model including an initial three-dimensional model and initial environmental information of a target area is constructed in a three-dimensional world server. After a business client sends a business service request for the target area to the three-dimensional world server, the three-dimensional world server sends the initial digital twin model of the initial three-dimensional model and the initial environmental information to the business client. The business client then corrects the initial three-dimensional model and the initial environmental information in combination with the sensor data collected by the sensor to obtain a real-time target three-dimensional model and target environmental information of the target area. Further, business operations can be performed according to the target three-dimensional model and the target environmental information, thereby realizing business operation decisions through a real-time digital twin three-dimensional model including a three-dimensional model and environmental information of the target area. The sensors required by the business client can be reduced, and the cost can be reduced. For example, the use of a bridge three-dimensional model and environmental information for bridge inspection can reduce the use of various sensors. In business fields such as intelligent logistics and autonomous driving, the digital twin three-dimensional model of the target area can be used for path planning, which is not limited by the detection distance of the vehicle-mounted radar of intelligent logistics and autonomous driving, thereby improving the field of view of intelligent logistics vehicles and autonomous driving vehicles and increasing the driving speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0051] Figure 1 A flowchart of a digital twin-based business service method provided by an embodiment of the present invention is shown.

[0052] Figure 2 A flowchart of a digital twin-based business service method provided by another embodiment of the present invention is shown.

[0053] Figure 3 A schematic diagram of the architecture of a business service system is shown;

[0054] Figure 4 An application architecture diagram of a digital twin-based business service system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Figure 1The flowchart of the business service method based on digital twins provided by an embodiment of the present invention is shown. Specifically, the business service method based on digital twins in an embodiment of the present invention specifically includes the following steps:

[0057] Step S101: The three-dimensional world server constructs an initial digital twin model of the target area, where the initial digital twin model includes an initial three-dimensional model and initial environmental information of the target area.

[0058] In this embodiment, the business service may be a service applied to three-dimensional data. For example, the business service may include one of an intelligent inspection business service, an intelligent logistics distribution business service, an assisted driving business service, a disaster emergency and warning business server, and a virtual reality business service.

[0059] Taking the inspection business service network as an example of bridge inspection business, the bridge inspection business needs to apply the three-dimensional data of the bridge, the three-dimensional data of vehicles traveling on the bridge, and the three-dimensional data of objects such as roads, mountains, rivers, etc. around the bridge. In the intelligent logistics distribution business, it is applied to the three-dimensional data of urban streets in the distribution business area, and in the assisted driving business, it is applied to the three-dimensional model of the city. The disaster emergency and early warning business is applied to the three-dimensional model of the disaster area before and after the disaster.

[0060] The three-dimensional world server can be a server that stores and maintains the initial digital twin model of the target area. The initial digital twin model includes an initial three-dimensional model and initial environmental information. The initial three-dimensional model can be a model constructed by points, lines, surfaces, bodies and other graphic elements. The initial environmental information may include temperature, humidity, weather, noise and other information. The environmental information may be different for different business services. Technical personnel in this field can set different environmental information according to business service requirements.

[0061] In this embodiment, the target area can be regularly surveyed and mapped by drones to generate an initial three-dimensional model of the target area, and the initial three-dimensional model can be updated after each business client uploads a real-time three-dimensional model of the target area. Similarly, the initial environmental information can be updated after each business client collects environmental information in the target area.

[0062] It should be noted that the initial three-dimensional model and environmental information of the target area in the three-dimensional world server can be generated after collecting data through sensors set in the target area. Taking the target area as a bridge as an example, the bridge can be regularly surveyed by drones to generate a time-stamped three-dimensional model of the bridge. Taking the target area as a city street scene as an example, a three-dimensional model of the city street scene can be generated through cameras on city streets, drone aerial surveys, remote sensing data, etc.

[0063] Step S102: the business client sends a business service request for a target area to the 3D world server.

[0064] The business client can be a client that performs business service operations. Before performing business service operations or during the process of performing business server operations, the business client can send a business service request for a target area to the three-dimensional world server. Taking the assisted autonomous driving business as an example, the assisted autonomous driving or its background server can send a path planning request from the current position to the target position to the three-dimensional world server after determining the target position and current position of the assisted autonomous driving vehicle, so as to request the three-dimensional world server to send a three-dimensional model of relevant roads and city street scenes from the current position to the target position.

[0065] Step S103: After receiving the business service request, the 3D world server sends the initial 3D model and initial environment information of the target area to the business client.

[0066] Specifically, the initial three-dimensional model and initial environment information of the target area are marked with time in the three-dimensional world server. The time indicates the time when the initial three-dimensional model and initial environment information are generated. After receiving the business service request, the three-dimensional world server searches for the latest initial three-dimensional model and initial environment information through the time in the business service request, and sends the initial three-dimensional model and initial environment information to the business client.

[0067] Step S104: the service client receives the initial three-dimensional model and initial environment information.

[0068] After the 3D world server sends the initial 3D model and initial environment information of the target area, the service client can receive the initial 3D model and initial environment information of the target area.

[0069] Step S105 : The service client receives the sensing data collected by the sensor, and reconstructs the initial three-dimensional model and updates the initial environment information according to the sensing data, so as to obtain the target three-dimensional model and target environment information of the target area.

[0070] In one embodiment, the service client is connected to one or more sensors to collect sensor data through the sensors, and step S105 may include the following steps:

[0071] S1051, the business client receives current environment information collected by the environment sensor, and receives surveying and mapping data collected by multiple surveying and mapping sensors;

[0072] S1052, preprocessing the current environment information, and determining the preprocessed environment information as target environment information of the target area;

[0073] S1053, preprocessing the surveying and mapping data collected by the plurality of surveying and mapping sensors to obtain preprocessed surveying and mapping data;

[0074] S1054, fusing the pre-processed surveying and mapping data to obtain fused data;

[0075] S1055, using the fused data to correct the initial three-dimensional model to obtain a target three-dimensional model of the target area.

[0076] Specifically, after receiving the initial three-dimensional model and initial environmental information of the target area, the business client simultaneously receives the current environmental information collected by the environmental sensors at the target area site, and receives the surveying and mapping data collected by multiple surveying and mapping sensors, where the environmental sensors may include sensors such as temperature, humidity, and noise, and the surveying and mapping sensors may include lidar, depth cameras, infrared sensors, ultrasonic sensors, inertial measurement units, satellite navigation systems, spectral sensors, pressure and tactile sensors, etc. The surveying and mapping data collected by each surveying and mapping sensor is deduplicated, deduplicated outliers, and filtered to obtain preprocessed surveying and mapping data, and the preprocessed surveying and mapping data are further fused to obtain fused data. The initial three-dimensional model is corrected through the fused data to obtain the target three-dimensional model.

[0077] Taking the surveying and mapping sensors including radar and camera as an example, the surveying and mapping data collected by the radar includes a first point cloud, and the surveying and mapping data of the camera is an image. Then, a second point cloud can be generated according to the preprocessed image, and the first point cloud and the second point cloud are fused to obtain a fused point cloud, and multiple images are spliced ​​to obtain a spliced ​​image. The fused point cloud and the initial three-dimensional model are input into the entity object recognition model to obtain a differentiated entity. According to the point cloud of the differentiated entity in the fused point cloud, the mesh of the differentiated entity is corrected in the initial three-dimensional model to obtain a corrected three-dimensional model. The spliced ​​image is used as a texture map to perform texture rendering on the corrected three-dimensional model to obtain a target three-dimensional model of the target area.

[0078] Among them, point cloud fusion can be fused by point cloud registration algorithms such as ICP (Iterative closest point) and NDT (Normal Distributions Transform), and images can be spliced ​​by algorithms such as SIFT (Scale-invariant feature transform) and SURF (Speeded Up Robust Features), and various pre-trained R-CNN and CNN neural networks can be used as entity object recognition models to identify differentiated entities in the fused point cloud and the initial three-dimensional model through the entity object recognition model, so as to re-model the differentiated entities in the initial three-dimensional model to obtain a corrected three-dimensional model. For example, taking assisted driving as an example, when the assisted driving vehicle travels to position A, after obtaining the initial three-dimensional model of position A from the three-dimensional world server, the point cloud is collected and fused by multiple sensors on the assisted driving vehicle to obtain a fused point cloud, and then the initial three-dimensional model and the fused point cloud are input into the entity object recognition model to obtain the differentiated entities between the initial three-dimensional model and the fused point cloud. , the differentiated entities may include a first type of entity that exists in the initial 3D model but does not exist in the fused point cloud, such as a vehicle that exists in the initial 3D model but does not exist in the fused point cloud, and also include a second type of entity that does not exist in the initial 3D model but exists in the fused point cloud. The first type of entity can be removed from the initial 3D model and the second type of entity can be added. Specifically, the first type of entity is removed from the initial 3D model and the surface is filled, and the point cloud of the second type of entity is used to generate triangular surfaces in the initial 3D model. Finally, the texture in the spliced ​​image is used to map and render the corrected 3D model to obtain a target 3D model of the target area. The point cloud registration, fusion, model correction and texture mapping rendering can refer to the existing technology, which will not be described in detail in this embodiment.

[0079] Step S106: The business client performs business operations according to the target three-dimensional model and target environment information.

[0080] After the initial three-dimensional model and initial environmental information are corrected to obtain the target three-dimensional model and target environmental information of the target area, business operations can be performed according to the target three-dimensional model and target environmental information. Specifically, initial business operation information can be loaded, and the initial business operation information is used to instruct the business client to perform business operations. The initial business operation information is corrected according to the target three-dimensional model, target environmental information, and preset business algorithms to obtain target business operation information, and business operations are performed according to the target business operation information. Different business services have different business operations, and their correction methods are also different. This embodiment does not limit the method of performing business operations according to the target three-dimensional model and target environmental information.

[0081] like Figure 2 As shown, in an optional embodiment, after S106, the following steps are further included:

[0082] S107, the business client uploads the target three-dimensional model and the target environment information as the target digital twin model of the target area to the three-dimensional world server;

[0083] S108, the three-dimensional world server uses the target digital twin model to replace the initial digital twin model as the updated initial digital twin model.

[0084] Through S107-S108, the digital twin model of the target area in the three-dimensional world server can be kept up to date.

[0085] In order to enable those skilled in the art to more clearly understand the business service method based on digital twins in the embodiment of the present invention, the following Figure 3 The following are some examples of specific business services:

[0086] like Figure 3 As shown, the business service system includes a business client and a three-dimensional world server, wherein the three-dimensional world server stores three-dimensional models and environmental information with time stamps, and the business client can be a client for business services such as intelligent inspection, intelligent logistics, autonomous driving, augmented reality, disaster emergency and early warning, etc. On the business client side, according to the business service requirements, one or more sensors including lidar, depth camera, infrared sensor, ultrasonic sensor, inertial measurement unit, satellite navigation system, spectral sensor, pressure and tactile sensor, and environmental sensor are connected, and data acquisition and preprocessing algorithms, point cloud processing algorithms, image processing and computer vision processing algorithms, three-dimensional reconstruction algorithms, data analysis and simulation algorithms, and real-time rendering and visualization algorithms are configured.

[0087] After the business client sends a business request to the 3D world server, the 3D world server sends the requested service (path planning, disaster warning, etc.) as well as the 3D model and environmental information to the business client. After receiving the data, the business client receives data from the sensor, updates the sent 3D model and environmental information through the configured algorithm, and obtains the corrected digital twin model including the 3D model and environmental information. The digital twin model is uploaded to the 3D world server to update the 3D model and environmental information in the 3D world server.

[0088] Taking bridge inspection as an example of intelligent inspection, in the prior art, it is necessary to install stress sensors, strain gauges, and accelerometers on the bridge to inspect and monitor the bridge. However, through the digital twin-based business service method of the embodiment of the present invention, the initial three-dimensional model and initial environmental information of the bridge are generated through the scanning data of the bridge inspection regularly collected by the drone and the environmental data of the surrounding roads or mountains. The images, point clouds, GPS location information and other data of the vehicles passing on the bridge are collected in real time through the cameras, lidars, etc. on the bridge. These data are integrated with the initial three-dimensional model and initial environmental information in the three-dimensional world server to obtain the target three-dimensional model and environmental information of the bridge and the surrounding area of ​​the bridge. Then, the target three-dimensional model and environmental information of different time periods are used to perform the inspection and judgment business of whether the status of the bridge is normal. After applying the solution of the present application, the bridge inspection only requires the drone to perform regular scanning along the specified route to obtain the scanning data of the key parts of the bridge. At the same time, combined with the data shared by the cameras, lidars and other sensors on the bridge, high-precision bridge inspection work can be completed, reducing the application and maintenance costs of various sensors and improving inspection efficiency.

[0089] In one example, taking the drone delivery scenario in intelligent logistics as an example, the existing drone delivery is that the drone delivers according to the pre-planned delivery route. During the delivery process, the trajectory is corrected by various three-dimensional perception sensors on the drone. The delivery route is relatively fixed, and the flexibility and efficiency are low. However, through the digital twin-based business service method of the embodiment of the present invention, the initial three-dimensional model of the delivery city area issued by the three-dimensional world server can be corrected through data collected by vehicles traveling in the city, data collected by street cameras, data from drone inspections, satellite remote sensing data, etc. The delivery route of the drone can be dynamically planned in advance through the corrected three-dimensional model of the city, and the drone delivery service can be carried out with a safer and more efficient delivery route. In addition, since the three-dimensional model is corrected by combining data collected by vehicles traveling in the city, data collected by street cameras, data from drone inspections, satellite remote sensing data, etc., there is no need to repeatedly configure the corresponding sensors on the drone, and the drone can use lower-cost sensors to achieve more efficient delivery services.

[0090] In another example, taking autonomous driving as an example, the existing autonomous driving is to use GPS to locate the autonomous driving vehicle and then perform path planning, and use the image, LIADR, RADAR and other sensors on the autonomous driving vehicle to collect data and then perform path correction to achieve a relatively stable autonomous driving service. However, in the existing autonomous driving, the farthest recognition distance of sensors such as image, LIADR, RADAR is usually 250 meters, and the expensive long-distance sensing unit will not exceed 500 meters, which seriously limits the speed of autonomous driving. However, through the digital twin-based business service method of the embodiment of the present invention, the three-dimensional model of the city can be corrected through the data collected by vehicles traveling in the city, the data collected by street cameras, the data of drone inspections, satellite remote sensing data, etc., and the specific position of the autonomous driving vehicle in the city and the three-dimensional information of the surrounding objects can be obtained. The three-dimensional information of all objects on the driving trajectory of the autonomous driving vehicle (including the three-dimensional information of objects 500 meters away) can be obtained through the three-dimensional world server, so that more advanced driving trajectory correction can be performed, and the speed of autonomous driving can be improved under the premise of ensuring safety.

[0091] In another example, taking earthquake emergency response in disaster emergency as an example, the existing solution is to find the locations of the hardest-hit areas through remote sensing data, low-altitude drone reconnaissance data and manual judgment, and conduct personnel search and rescue, resulting in insufficient emergency response capabilities and difficulty in locating the required search and rescue points within the golden 72 hours. However, through the digital twin-based business service method of the embodiment of the present invention, remote sensing data and low-altitude drone reconnaissance data are compared with the regional data before the earthquake in the three-dimensional world server, and the locations of the hardest-hit areas can be automatically determined immediately, and search and rescue points and search and rescue tasks can be formulated based on this.

[0092] Taking dynamic augmented reality (AR) as an example, the existing solution is to wear a bulky AR module with a large number of sensor units integrated inside to enhance the feeling of the virtual reality world. However, through the digital twin-based business service method of the embodiment of the present invention, the position of the experiencer in the system and the surrounding three-dimensional model are obtained through the three-dimensional world server, and the AR module does not need to be configured with various sensors to reconstruct the three-dimensional model. AR equipment manufacturers can not only simplify the equipment structure, but also obtain more realistic information about the area, which can greatly enhance the user's AR experience.

[0093] The present invention is based on digital twin technology. An initial digital twin model including an initial three-dimensional model and initial environmental information of a target area is constructed in a three-dimensional world server. After a business client sends a business service request for the target area to the three-dimensional world server, the three-dimensional world server sends the initial digital twin model of the initial three-dimensional model and the initial environmental information to the business client. The business client then corrects the initial three-dimensional model and the initial environmental information in combination with the sensor data collected by the sensor to obtain a real-time target three-dimensional model and target environmental information of the target area. Further, business operations can be performed according to the target three-dimensional model and the target environmental information, thereby realizing business operation decisions through a real-time digital twin three-dimensional model including a three-dimensional model and environmental information of the target area. The sensors required by the business client can be reduced, and the cost can be reduced. For example, the use of a bridge three-dimensional model and environmental information for bridge inspection can reduce the use of various sensors. In business fields such as intelligent logistics and autonomous driving, the digital twin three-dimensional model of the target area can be used for path planning, which is not limited by the detection distance of the vehicle-mounted radar of intelligent logistics and autonomous driving, thereby improving the field of view of intelligent logistics vehicles and autonomous driving vehicles and increasing the driving speed.

[0094] Figure 4 An application architecture diagram of a digital twin-based business service system provided in an embodiment of the present invention is shown. The digital twin-based business service system in this embodiment specifically includes a three-dimensional world server 40 and a business client 41. The three-dimensional world server 40 includes a digital twin model maintenance unit 401 and a business service request response unit 402. The business client 41 includes a business request unit 411, an initial digital twin model receiving unit 412, a digital twin model reconstruction unit 413, and a business execution unit 414.

[0095] The digital twin model maintenance unit 401 in the three-dimensional world server 40 is used to construct an initial digital twin model of the target area, wherein the initial digital twin model includes an initial three-dimensional model and initial environment information of the target area;

[0096] The service request unit 411 in the service client 41 is used to send a service request for the target area to the three-dimensional world server;

[0097] The business service request response unit 402 in the three-dimensional world server 40 is used to send the initial three-dimensional model and initial environment information of the target area to the business client after receiving the business service request;

[0098] The initial digital twin model receiving unit 412 in the business client 41 is used to receive the initial three-dimensional model and initial environment information;

[0099] The digital twin model reconstruction unit 413 in the business client 41 is used to receive the sensor data collected by the sensor, and reconstruct the initial three-dimensional model and update the initial environment information according to the sensor data to obtain the target three-dimensional model and target environment information of the target area;

[0100] The service execution unit 414 in the service client 41 is used to execute service operations according to the target three-dimensional model and the target environment information.

[0101] As a further limitation of the embodiment of the present invention, the service client 41 further includes:

[0102] A model uploading unit, configured to upload the target three-dimensional model and target environment information as a target digital twin model of the target area to the three-dimensional world server;

[0103] The three-dimensional world server 40 further includes:

[0104] A digital twin model updating unit is used to replace the initial digital twin model with the target digital twin model as the updated initial digital twin model.

[0105] As a further limitation of the embodiment of the present invention, the digital twin model reconstruction unit 413 specifically includes the following modules:

[0106] A sensor data receiving module is used to receive current environmental information collected by environmental sensors, and receive surveying and mapping data collected by multiple surveying and mapping sensors;

[0107] An environmental information preprocessing module, used to preprocess the current environmental information and determine the preprocessed environmental information as the target environmental information of the target area;

[0108] A surveying and mapping data preprocessing module is used to preprocess the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data;

[0109] A data fusion module is used to fuse the pre-processed surveying and mapping data to obtain fused data;

[0110] The three-dimensional model correction module is used to correct the initial three-dimensional model using the fusion data to obtain a target three-dimensional model of the target area.

[0111] As a further limitation of the embodiment of the present invention, the surveying and mapping data preprocessing module is specifically used to perform deduplication, deduplication outlier and filtering processing on the surveying and mapping data collected by each surveying and mapping sensor to obtain preprocessed surveying and mapping data.

[0112] As a further limitation of the embodiment of the present invention, the data fusion module is specifically used for:

[0113] Generate a second point cloud according to the preprocessed image, and fuse the first point cloud and the second point cloud to obtain a fused point cloud;

[0114] Multiple images are stitched together to obtain a stitched image.

[0115] As a further limitation of the embodiment of the present invention, the three-dimensional model correction module is specifically used for:

[0116] Inputting the fused point cloud and the initial three-dimensional model into a physical object recognition model to obtain a differentiated entity;

[0117] Correcting the mesh of the differentiated entity in the initial three-dimensional model according to the point cloud of the differentiated entity in the fused point cloud to obtain a corrected three-dimensional model;

[0118] The spliced ​​image is used as a texture map to perform texture rendering on the corrected three-dimensional model to obtain a target three-dimensional model of the target area.

[0119] As a further limitation of the embodiment of the present invention, the service execution unit 414 specifically includes the following modules:

[0120] A loading module, used to load initial business operation information, wherein the initial business operation information is used to instruct the business client to perform a business operation;

[0121] An operation information correction module, used to correct the initial business operation information according to the target three-dimensional model, the target environment information and a preset business algorithm to obtain target business operation information;

[0122] The business operation execution module is used to execute the business operation according to the target business operation information.

[0123] As a further limitation of an embodiment of the present invention, the business service includes one of an intelligent inspection business service, an intelligent logistics distribution business service, an assisted driving business service, a disaster emergency and warning business server, and a virtual reality business service.

[0124] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0125] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0126] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

[0128] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A business service method based on digital twins, characterized in that: The business service system is applied to a 3D world server and a business client, and specifically includes the following steps: The three-dimensional world server constructs an initial digital twin model of the target area, wherein the initial digital twin model includes an initial three-dimensional model and initial environmental information of the target area, wherein the initial environmental information includes temperature, humidity, weather, and noise; The business client sends a business service request for the target area to the three-dimensional world server; After receiving the business service request, the 3D world server sends the initial 3D model and initial environment information of the target area to the business client; The service client receives the initial three-dimensional model and initial environment information; The service client receives the sensor data collected by the sensor, and reconstructs the initial three-dimensional model and updates the initial environment information according to the sensor data to obtain the target three-dimensional model and target environment information of the target area; The business client performs business operations according to the target three-dimensional model and the target environment information; The service client receives the sensor data collected by the sensor, and reconstructs the initial three-dimensional model and updates the initial environment information according to the sensor data to obtain the target three-dimensional model and target environment information of the target area, specifically including the following steps: The service client receives current environmental information collected by the environmental sensor, and receives surveying and mapping data collected by multiple surveying and mapping sensors; Preprocessing the current environmental information, and determining the preprocessed environmental information as the target environmental information of the target area; Preprocessing the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data; Fusing the preprocessed surveying and mapping data to obtain fused data; The initial three-dimensional model is modified using the fusion data to obtain a target three-dimensional model of the target area.

2. The digital twin-based business service method according to claim 1, characterized in that: The following steps are also included: The business client uploads the target three-dimensional model and target environment information as a target digital twin model of the target area to the three-dimensional world server; The three-dimensional world server uses the target digital twin model to replace the initial digital twin model as the updated initial digital twin model.

3. The digital twin-based business service method according to claim 1, characterized in that: Preprocessing the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data specifically includes the following steps: The surveying and mapping data collected by each surveying and mapping sensor are deduplicated, deduplicated outliers are removed, and filtered to obtain preprocessed surveying and mapping data.

4. The digital twin-based business service method according to claim 1, characterized in that: The surveying and mapping data includes multiple images and multiple first point clouds. The pre-processed surveying and mapping data are fused to obtain fused data, which specifically includes the following steps: Generate a second point cloud according to the preprocessed image, and fuse the first point cloud and the second point cloud to obtain a fused point cloud; Multiple images are stitched together to obtain a stitched image.

5. The digital twin-based business service method according to claim 4, characterized in that: The initial three-dimensional model is modified by using the fusion data to obtain a target three-dimensional model of the target area, which specifically includes the following steps: Inputting the fused point cloud and the initial three-dimensional model into a physical object recognition model to obtain a differentiated entity; Correcting the mesh of the differentiated entity in the initial three-dimensional model according to the point cloud of the differentiated entity in the fused point cloud to obtain a corrected three-dimensional model; The spliced ​​image is used as a texture map to perform texture rendering on the corrected three-dimensional model to obtain a target three-dimensional model of the target area.

6. The digital twin-based business service method according to any one of claims 1 to 5, characterized in that: The business client performs a business operation according to the target three-dimensional model and the target environment information, specifically comprising the following steps: Loading initial business operation information, where the initial business operation information is used to instruct the business client to perform a business operation; Modifying the initial business operation information according to the target three-dimensional model, the target environment information and a preset business algorithm to obtain target business operation information; Execute a business operation according to the target business operation information.

7. The digital twin-based business service method according to claim 6, characterized in that: The business services include one of intelligent inspection business services, intelligent logistics distribution business services, assisted driving business services, disaster emergency and early warning business servers, and virtual reality business services.

8. A business service system based on digital twins, characterized in that: Specifically, it includes a three-dimensional world server and a business client. The three-dimensional world server includes a digital twin model maintenance unit and a business service request response unit. The business client includes a business request unit, an initial digital twin model receiving unit, a digital twin model reconstruction unit and a business execution unit. The digital twin model maintenance unit in the three-dimensional world server is used to construct an initial digital twin model of the target area, wherein the initial digital twin model includes an initial three-dimensional model and initial environmental information of the target area; The service request unit in the service client is used to send a service request for the target area to the three-dimensional world server; The business service request response unit in the three-dimensional world server is used to send the initial three-dimensional model and initial environment information of the target area to the business client after receiving the business service request; The initial digital twin model receiving unit in the business client is used to receive the initial three-dimensional model and initial environmental information, where the initial environmental information includes temperature, humidity, weather, and noise; The digital twin model reconstruction unit in the business client is used to receive the sensor data collected by the sensor, and reconstruct the initial three-dimensional model and update the initial environmental information according to the sensor data to obtain the target three-dimensional model and target environmental information of the target area; The service execution unit in the service client is used to execute service operations according to the target three-dimensional model and the target environment information; The digital twin model reconstruction unit specifically includes the following modules: A sensor data receiving module is used to receive current environmental information collected by environmental sensors, and receive surveying and mapping data collected by multiple surveying and mapping sensors; An environmental information preprocessing module, used to preprocess the current environmental information and determine the preprocessed environmental information as the target environmental information of the target area; A surveying and mapping data preprocessing module is used to preprocess the surveying and mapping data collected by multiple surveying and mapping sensors to obtain preprocessed surveying and mapping data; A data fusion module is used to fuse the pre-processed surveying and mapping data to obtain fused data; The three-dimensional model correction module is used to correct the initial three-dimensional model using the fusion data to obtain a target three-dimensional model of the target area.

9. The digital twin-based business service system according to claim 8, characterized in that: The business client also includes: A model uploading unit, configured to upload the target three-dimensional model and target environment information as a target digital twin model of the target area to the three-dimensional world server; The three-dimensional world server also includes: A digital twin model updating unit is used to replace the initial digital twin model with the target digital twin model as the updated initial digital twin model.