Multi-source data fusion enhancement method and device

Abstract

The invention discloses a multi-source data fusion enhancement method and device. The method comprises the following steps: acquiring the position and field environment data of augmented reality equipment, and establishing a three-dimensional environment model; identifying target identification information related to the network element in the field environment data, and retrieving a matched target network element in a service system database to obtain the position, the working state and the service data of the target network element; acquiring map data from a map database, determining a spatial congestion state from the network element to the device position, and predicting a signal state of the device position in combination with the working state of the target network element; and fusing the position, the working state, the service data and the predicted signal state of the network element into a three-dimensional environment model to form a comprehensive environment model, and sending the comprehensive environment model to augmented reality equipment for display. The technical problems that in a network element operation and maintenance scene, information of multiple data sources is difficult to comprehensively utilize, and the operation and maintenance processing efficiency is low are solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a multi-source data fusion enhancement method and device. Background Art

[0002] From the perspective of technical implementation, augmented reality, virtual reality and low-latency communication are the core elements that promote the development of virtual technologies such as metaverse and digital twins. However, the development of existing virtual technologies also faces significant challenges. At present, key technologies such as augmented reality, virtual reality, and artificial intelligence are mostly deployed independently, and have not been deeply integrated, making it difficult to meet the needs of multi-source data enhancement presentation in complex tasks such as mobile communication network optimization and planning. For example, although augmented reality glasses can display general environmental information such as road surface information or surrounding buildings through geographic environment modeling, in actual applications, users often need more comprehensive information, such as the presence of base stations / cells in the environment and their signal strength and service data. The coverage, performance indicators, user complaints, road test results or simulation data of 5G NR (NewRadio) cells are superimposed on the terrain and objects of augmented reality glasses to provide a deeper environmental perception. Therefore, in order to give full play to the potential of virtual technology, it is necessary to develop an enhanced processing method that can fuse multi-source heterogeneous data to enhance the value and effect of virtual technology in actual application scenarios.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a multi-source data fusion enhancement method and device to at least solve the technical problem that it is difficult to comprehensively utilize information from multiple data sources in network element operation and maintenance scenarios and the operation and maintenance processing efficiency is low.

[0005] According to one aspect of an embodiment of the present application, a multi-source data fusion enhancement method is provided, which is applied to a network element business system, including: obtaining a device position of an augmented reality device and field environment data collected by the augmented reality device, and establishing a three-dimensional environment model based on the field environment data; identifying target identification information related to the network element in the field environment data, and retrieving the target identification information from a business system database, if a target network element matching the target identification information is retrieved, obtaining the network element position, working status and business data of the target network element; obtaining map data from a map database, and determining a spatial congestion status from the network element position to the device position based on the map data, and predicting a signal status of the device position based on the working status and spatial congestion status of the target network element; fusing the network element position, working status, business data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model; and sending the comprehensive environment model to the augmented reality device for display.

[0006] Optionally, the device position of the augmented reality device and the on-site environment data collected by the augmented reality device are obtained, and a three-dimensional environment model is established based on the on-site environment data, including: positioning the augmented reality device to obtain the device position of the augmented reality device; obtaining the on-site environment data collected by a depth image acquisition device integrated in the augmented reality device, wherein the on-site environment data is depth image data; and establishing a three-dimensional environment model based on the on-site environment data, wherein the types of three-dimensional environment models include: three-dimensional point cloud maps.

[0007] Optionally, identifying target identification information related to network elements in field environment data includes: calling a pre-trained image recognition model to identify the field environment data, and determining target identification information related to network elements in the field environment data, wherein the type of network element includes at least one of the following: base station, cell, and the type of target identification information includes at least one of the following: installation location information and morphology information of the network element.

[0008] Optionally, the target identification information is retrieved from the business system database, including: comparing the target identification information with the network element identification information of the corresponding type of each network element stored in the business system database; determining that the network element whose network element identification information is the same as the target identification information, or the difference meets a preset condition, is the target network element that matches the target identification information.

[0009] Optionally, the map data includes: a geographic information system map, determining the spatial congestion state from the network element location to the device location based on the map data, including: determining a vector line segment from the network element location to the device location in the geographic information system map, and determining whether the vector line segment crosses the terrain; if the vector line segment crosses the terrain, determining that the spatial congestion state is congested; if the vector line segment does not cross the terrain, determining that the spatial congestion state is unblocked.

[0010] Optionally, the working status of the target network element includes: output signal strength, predicting the signal status of the device location based on the working status of the target network element and the spatial congestion status, including: when the spatial congestion status is congested, determining the type of terrain and objects crossed by the vector line segment, and determining the signal status of the device location based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the corresponding type of terrain and objects, and the distance from the network element location to the device location, wherein the signal status includes at least one of the following: received signal strength at the device location, and signal coverage status within a preset range around the device location; when the spatial congestion status is unobstructed, determining the signal status of the device location based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the air, and the distance from the network element location to the device location.

[0011] Optionally, the network element service system is deployed on: a mobile device directly connected to the augmented reality device via a data cable, or a cloud server that transmits data with the augmented reality device via a wireless network.

[0012] According to another aspect of an embodiment of the present application, a multi-source data fusion enhancement device is also provided, including: an acquisition module, used to acquire the device position of an augmented reality device and the on-site environment data collected by the augmented reality device, and establish a three-dimensional environment model based on the on-site environment data; an identification module, used to identify target identification information related to the network element in the on-site environment data, and retrieve the target identification information from the business system database, if a target network element matching the target identification information is retrieved, the network element position, working status and business data of the target network element are acquired; a prediction module, used to acquire map data from a map database, and determine the spatial congestion status from the network element position to the device position based on the map data, and predict the signal status of the device position based on the working status and spatial congestion status of the target network element; a fusion module, used to fuse the network element position, working status, business data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model; a display module, used to send the comprehensive environment model to the augmented reality device for display.

[0013] According to another aspect of an embodiment of the present application, a computer program product is further provided, the computer program product comprising: a computer program, wherein when the computer program is executed by a processor, the multi-source data fusion enhancement method described above is implemented.

[0014] According to another aspect of an embodiment of the present application, an electronic device is further provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned multi-source data fusion enhancement method through the computer program.

[0015] In the embodiment of the present application, by acquiring the device location and on-site environment data of the augmented reality device, an accurate three-dimensional environment model is constructed, which provides a real and reliable environment background for subsequent network element identification and signal state prediction. By identifying the identification information related to a specific network element in the on-site environment data and comparing it with the business system database, the network element information matching the identification information can be retrieved. The spatial congestion status of the signal transmission path is evaluated based on the map data, and the signal status of the device location is predicted in combination with the working status of the network element and the environmental congestion. This method takes into account the impact of the physical environment and the working status of the network element on the signal quality, and can provide users with more accurate signal predictions. Furthermore, the detailed information of the network element and the signal status prediction results are integrated with the three-dimensional environmental model to generate a comprehensive environmental model including the network element location, working status, service data and signal status, thereby realizing the seamless integration of multi-source data, allowing engineers to intuitively see the key information such as the distribution, status and signal quality of the network element in an integrated view, greatly improving the efficiency and accuracy of network operation and maintenance work, and sending the comprehensive environmental model to the augmented reality device for display, so that engineers can directly see the comprehensive view superimposed with network element information and signal status through the augmented reality device in the on-site environment, thereby obtaining a more intuitive and comprehensive network status perception in actual operation, facilitating rapid problem location and optimization of network performance, and solving the technical problems of difficulty in comprehensively utilizing information from multiple data sources and low operation and maintenance processing efficiency in network element operation and maintenance scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a flowchart of an optional multi-source data fusion enhancement method according to an embodiment of the present application;

[0018] Figure 2 is a structural schematic diagram of an optional multi-source data fusion enhancement device according to an embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Example 1

[0023] According to an embodiment of the present application, a multi-source data fusion enhancement method is provided, which is applied to a network element service system. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of a multi-source data fusion enhancement method provided according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0025] Step S102, obtaining the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and establishing a three-dimensional environment model based on the on-site environment data;

[0026] Step S104, identifying target identification information related to the network element in the field environment data, and retrieving the target identification information from the service system database, and if a target network element matching the target identification information is retrieved, obtaining the network element location, working status and service data of the target network element;

[0027] Step S106, obtaining map data from a map database, and determining a spatial congestion state from the network element location to the device location based on the map data, and predicting a signal state at the device location based on a working state of the target network element and the spatial congestion state;

[0028] Step S108, integrating the network element position, working status, service data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model;

[0029] Step S110, sending the comprehensive environment model to the augmented reality device for display.

[0030] The following describes the various steps of the multi-source data fusion enhancement method in combination with a specific implementation process.

[0031] First, the device position of the augmented reality device and the on-site environment data collected by the augmented reality device are obtained, and a three-dimensional environment model is established based on the on-site environment data. This process can be carried out in the following manner:

[0032] Positioning the augmented reality device to obtain the device position of the augmented reality device; obtaining the on-site environment data collected by the depth image acquisition device integrated in the augmented reality device, wherein the on-site environment data is depth image data; establishing a three-dimensional environment model based on the on-site environment data, wherein the types of the three-dimensional environment model include: three-dimensional point cloud map.

[0033] Specifically, it is necessary to first determine the precise location of the augmented reality device (such as AR (Augmented Reality) glasses) in the real world. The location of the device can be obtained through a variety of positioning technologies, such as GPS (Global Positioning System), base station positioning, Wi-Fi positioning or more advanced indoor positioning technology. This location information is crucial because it will be used for subsequent vector calculations and data fusion. Augmented reality devices usually integrate a variety of sensors and image acquisition devices, including depth cameras, which can capture the depth information of the scene. The on-site environmental data acquired by the depth image acquisition device is depth image data, which contains specific information about the distance between the object and the device, as well as the three-dimensional structure of the environment. After obtaining the depth image data, the next task is to convert it into a three-dimensional model, which usually involves the process of point cloud generation, in which the depth information of each pixel is converted into a point in three-dimensional space. These points are combined to form a three-dimensional point cloud map. The point cloud map not only includes the shape of environmental objects, but also their location information, which is extremely important for subsequent fusion calculations. The establishment of a three-dimensional environment model may also involve three-dimensional modeling and texture mapping to enhance the visual realism of the model. The model may be generated using off-the-shelf three-dimensional modeling software or a customized modeling algorithm, depending on the required accuracy and complexity.

[0034] After obtaining the field environment data, identify the target identification information related to the network element in the field environment data, and retrieve the target identification information from the business system database. If a target network element matching the target identification information is retrieved, obtain the network element location, working status and business data of the target network element.

[0035] As an optional implementation, identifying target identification information related to network elements in field environment data may include the following steps: calling a pre-trained image recognition model to identify the field environment data, and determining the target identification information related to the network elements in the field environment data. These models have been trained with a large amount of image data and can identify specific types of objects, including base stations, cells and other network elements of different forms. The model will focus on detecting the network element installation location information, form information, etc. in the field environment data. This information is crucial for determining the existence of the network element and its specific characteristics. Among them, the type of network element includes at least one of the following: base station, cell, and the type of target identification information includes at least one of the following: installation location information and form information of the network element.

[0036] As an optional implementation, the installation location information of the network element may include the building number, floor, latitude and longitude coordinates of the specific installation point, etc. of the building where the network element is located. This information helps to accurately determine the location of the network element in the environment. The morphological information of the network element may include the appearance features of the network element, such as the shape, size, and direction of the antenna. This information is equally important for subsequent network element matching and signal strength calculation.

[0037] After identifying the target identification information, the identified target identification information is matched with the network element information in the business system database to determine whether there is an entity in the field environment that matches the network element recorded in the database.

[0038] As an optional implementation, retrieving target identification information from a business system database may include the following steps: comparing the target identification information with the network element identification information of corresponding types of each network element stored in the business system database; determining that the network element identification information is the same as the target identification information, such as the same house number, or the difference meets preset conditions, such as similar shape, as the target network element that matches the target identification information.

[0039] Once the target network element that matches the identification information in the field environment data is determined, the detailed information of the network element can be retrieved from the business system database, including but not limited to: network element location, working status and business data, etc. The network element location includes the specific installation location of the network element, such as longitude and latitude coordinates, relative altitude, etc. The working status includes whether the network element is currently in working state, working mode, equipment health status, etc. The business data covers the performance indicators of the network element, coverage, number of users, complaint records, road test data or simulation data, etc. These data are crucial for network operation and optimization.

[0040] After determining the target network element, map data is obtained from the map database, and the spatial congestion status from the network element location to the device location is determined based on the map data. The signal status at the device location is predicted based on the working status of the target network element and the spatial congestion status. The working status of the network element includes information such as whether it is operating normally, signal transmission power, and working frequency band. These factors will directly affect the signal strength. The spatial congestion status provides a perspective on whether the signal transmission path is affected by the physical environment. Combined with the electromagnetic wave propagation model, the degree of signal attenuation before reaching the AR device can be more accurately predicted.

[0041] As an optional implementation, the map data includes: a GIS map. The spatial congestion status from the network element location to the device location is determined based on the map data. The following steps can be taken: determine the vector line segment from the network element location to the device location in the GIS map, and determine whether the vector line segment crosses the terrain. This line segment represents a potential transmission path for the wireless signal and is an intuitive way to evaluate the congestion status of the signal path; if the vector line segment crosses the terrain, the spatial congestion status is determined to be congested; if the vector line segment does not cross the terrain, the spatial congestion status is determined to be unblocked. This step can be achieved through the layer information and spatial analysis tools in the GIS map to determine the spatial relationship between the line segment and the terrain.

[0042] As an optional implementation manner, the working state of the target network element includes: output signal strength, and the signal state of the device location is predicted according to the working state of the target network element and the spatial congestion state, which can be performed in the following manner:

[0043] When the spatial congestion state is congested, determine the type of terrain and features that the vector line segment passes through, and determine the signal state of the device location based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the terrain and features of the corresponding type, and the distance from the network element location to the device location, wherein the signal state includes at least one of the following: the received signal strength at the device location, and the signal coverage state within a preset range around the device location;

[0044] When the spatial congestion status is unblocked, the signal status of the device location is determined based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the air, and the distance from the network element location to the device location.

[0045] After obtaining the above-mentioned network element position, working status, service data and signal status, the network element position, working status, service data and signal status of the target network element are integrated with the three-dimensional environment model to obtain a comprehensive environment model.

[0046] This method integrates the network element location, working status, business data and other information obtained from the business system database with the three-dimensional environment model established based on the on-site environment data. This means that the location of the network element is accurately identified in the three-dimensional model, and the working status of the network element (such as whether it is operating normally, signal transmission power, etc.) and business data (such as the number of users, performance indicators, complaint records, etc.) are superimposed on the model, so that the three-dimensional environment model not only contains environmental characteristics, but also incorporates the dynamic information of the network element. This model not only shows a three-dimensional view of the on-site environment, but also intuitively shows the distribution of network elements, working status and signal quality at the device location, providing engineers with a comprehensive network status view, which helps to more accurately perform network optimization, planning and other business work.

[0047] After the comprehensive environment model is obtained by the above method, the comprehensive environment model is sent to the augmented reality device for display.

[0048] As an optional implementation, the network element service system can be deployed on: a mobile device that is directly connected to the augmented reality device via a data cable, or a cloud server that transmits data to the augmented reality device via a wireless network.

[0049] In the embodiment of the present application, by acquiring the device location and on-site environment data of the augmented reality device, an accurate three-dimensional environment model is constructed, which provides a real and reliable environment background for subsequent network element identification and signal state prediction. By identifying the identification information related to a specific network element in the on-site environment data and comparing it with the business system database, the network element information matching the identification information can be retrieved. The spatial congestion status of the signal transmission path is evaluated based on the map data, and the signal status of the device location is predicted in combination with the working status of the network element and the environmental congestion. This method takes into account the impact of the physical environment and the working status of the network element on the signal quality, and can provide users with more accurate signal predictions. Furthermore, the detailed information of the network element and the signal status prediction results are integrated with the three-dimensional environmental model to generate a comprehensive environmental model including the network element location, working status, service data and signal status, thereby realizing the seamless integration of multi-source data, allowing engineers to intuitively see the key information such as the distribution, status and signal quality of the network element in an integrated view, greatly improving the efficiency and accuracy of network operation and maintenance work, and sending the comprehensive environmental model to the augmented reality device for display, so that engineers can directly see the comprehensive view superimposed with network element information and signal status through the augmented reality device in the on-site environment, thereby obtaining a more intuitive and comprehensive network status perception in actual operation, facilitating rapid problem location and optimization of network performance, and solving the technical problems of difficulty in comprehensively utilizing information from multiple data sources and low operation and maintenance processing efficiency in network element operation and maintenance scenarios.

[0050] Example 2

[0051] According to an embodiment of the present application, a multi-source data fusion enhancement device for implementing the multi-source data fusion enhancement method in Embodiment 1 is also provided. Figure 2 As shown, the multi-source data fusion enhancement device at least includes: an acquisition module 21, a recognition module 22, a prediction module 23, a fusion module 24 and a display module 25, wherein:

[0052] An acquisition module 21 is used to acquire the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and to establish a three-dimensional environment model based on the on-site environment data;

[0053] The identification module 22 is used to identify the target identification information related to the network element in the field environment data, and retrieve the target identification information from the business system database. If a target network element matching the target identification information is retrieved, the network element location, working status and business data of the target network element are obtained;

[0054] The prediction module 23 is used to obtain map data from the map database, determine the spatial congestion state from the network element location to the device location based on the map data, and predict the signal state of the device location based on the working state of the target network element and the spatial congestion state;

[0055] A fusion module 24 is used to fuse the network element position, working status, service data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model;

[0056] The display module 25 is used to send the comprehensive environment model to the augmented reality device for display.

[0057] The functions of each module of the multi-source data fusion enhancement device are explained below in conjunction with a specific implementation process.

[0058] The acquisition module acquires the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and establishes a three-dimensional environment model based on the on-site environment data. The following methods can be used:

[0059] Positioning the augmented reality device to obtain the device position of the augmented reality device; obtaining the on-site environment data collected by the depth image acquisition device integrated in the augmented reality device, wherein the on-site environment data is depth image data; establishing a three-dimensional environment model based on the on-site environment data, wherein the types of the three-dimensional environment model include: three-dimensional point cloud map.

[0060] The identification module identifies target identification information related to the network element in the field environment data, and retrieves the target identification information from the business system database. If a target network element matching the target identification information is retrieved, the network element location, working status and business data of the target network element are obtained.

[0061] As an optional implementation, identifying target identification information related to network elements in field environment data may include the following steps: calling a pre-trained image recognition model to identify the field environment data, and determining target identification information related to network elements in the field environment data, wherein the type of network element includes at least one of the following: base station, cell, and the type of target identification information includes at least one of the following: installation location information and morphology information of the network element.

[0062] As an optional implementation, retrieving target identification information from a business system database may include the following steps: comparing the target identification information with the corresponding type of network element identification information of each network element stored in the business system database; determining that the network element whose network element identification information is the same as the target identification information, or the difference meets a preset condition, is the target network element that matches the target identification information.

[0063] The prediction module obtains map data from the map database, determines the spatial congestion state from the network element location to the device location based on the map data, and predicts the signal state of the device location based on the working state of the target network element and the spatial congestion state.

[0064] As an optional implementation, the map data includes: a geographic information system map, determining the spatial congestion state from the network element location to the device location based on the map data, including: determining a vector line segment from the network element location to the device location in the geographic information system map, and determining whether the vector line segment crosses the terrain; if the vector line segment crosses the terrain, determining that the spatial congestion state is congested; if the vector line segment does not cross the terrain, determining that the spatial congestion state is unblocked.

[0065] As an optional implementation, the working status of the target network element includes: output signal strength, predicting the signal status of the device location based on the working status of the target network element and the spatial congestion status, including: when the spatial congestion status is congested, determining the type of terrain and objects crossed by the vector line segment, and determining the signal status of the device location based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the corresponding type of terrain and objects, and the distance from the network element location to the device location, wherein the signal status includes at least one of the following: received signal strength at the device location, and signal coverage status within a preset range around the device location; when the spatial congestion status is unobstructed, determining the signal status of the device location based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the air, and the distance from the network element location to the device location.

[0066] The fusion module fuses the network element location, working status, service data, signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model.

[0067] After obtaining the comprehensive environment model, the display module sends the comprehensive environment model to the augmented reality device for display.

[0068] As an optional implementation, the network element service system can be deployed on: a mobile device that is directly connected to the augmented reality device via a data cable, or a cloud server that transmits data to the augmented reality device via a wireless network.

[0069] It should be noted that each module in the multi-source data fusion enhancement device in the embodiment of the present application corresponds one by one to each implementation step of the multi-source data fusion enhancement method in Example 1. Since a detailed description has been given in Example 1, some details not reflected in this embodiment can be referred to Example 1 and will not be repeated here.

[0070] Example 3

[0071] According to an embodiment of the present application, a computer program product is further provided. The computer program product includes a computer program, wherein when the computer program is executed by a processor, the multi-source data fusion enhancement method in Embodiment 1 is implemented.

[0072] According to an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the multi-source data fusion enhancement method in Example 1 by running the computer program.

[0073] According to an embodiment of the present application, a processor is further provided, which is used to run a computer program, wherein the multi-source data fusion enhancement method in Example 1 is executed when the computer program is running.

[0074] According to an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the multi-source data fusion enhancement method in Example 1 through the computer program.

[0075] Specifically, the following steps are executed when the computer program is running: obtaining the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and establishing a three-dimensional environment model based on the on-site environment data; identifying the target identification information related to the network element in the on-site environment data, and retrieving the target identification information from the business system database, if the target network element matching the target identification information is retrieved, obtaining the network element position, working status and business data of the target network element; obtaining map data from the map database, and determining the spatial congestion status from the network element position to the device position based on the map data, and predicting the signal status of the device position based on the working status and spatial congestion status of the target network element; integrating the network element position, working status, business data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model; and sending the comprehensive environment model to the augmented reality device for display.

[0076] As an optional implementation, the electronic device may be in the form of a mobile terminal, a computer terminal or a similar computing device. Figure 3 FIG. 1 shows a hardware structure block diagram of an electronic device for implementing a multi-source data fusion enhancement method. Figure 3As shown, the electronic device 30 may include one or more (302a, 302b, ..., 302n are used to illustrate) processors 302 (the processor 302 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components as shown, or with Figure 3 Different configurations shown.

[0077] It should be noted that the one or more processors 302 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the electronic device 30. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0078] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the multi-source data fusion enhancement method in the embodiment of the present application. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, that is, to implement the vulnerability detection method of the above-mentioned application. The memory 304 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 304 may further include a memory remotely arranged relative to the processor 302, and these remote memories may be connected to the electronic device 30 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0079] The transmission device 306 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 30. In one example, the transmission device 306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0080] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 30 .

[0081] The serial numbers of the above embodiments are only for description and do not represent the advantages or disadvantages of the embodiments.

[0082] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0086] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0087] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multi-source data fusion enhancement method, applied to a network element service system, characterized in that: include: Acquire the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and establish a three-dimensional environment model according to the on-site environment data; Identify target identification information related to the network element in the field environment data, and retrieve the target identification information from the service system database, and if a target network element matching the target identification information is retrieved, obtain the network element location, working status and service data of the target network element; Acquire map data from a map database, determine a spatial congestion state from the network element location to the device location based on the map data, and predict a signal state at the device location based on a working state of the target network element and the spatial congestion state; The network element position, working status, service data, and signal status of the target network element are integrated with the three-dimensional environment model to obtain a comprehensive environment model; The comprehensive environment model is sent to the augmented reality device for display.

2. The method according to claim 1, characterized in that: Acquiring a device position of an augmented reality device and on-site environment data collected by the augmented reality device, and establishing a three-dimensional environment model according to the on-site environment data, including: Positioning the augmented reality device to obtain a device position of the augmented reality device; Acquire the scene environment data collected by the depth image acquisition device integrated in the augmented reality device, wherein the scene environment data is the depth image data; A three-dimensional environment model is established based on the on-site environment data, wherein the type of the three-dimensional environment model includes: a three-dimensional point cloud map.

3. The method according to claim 2, characterized in that Identifying target identification information related to a network element in the field environment data, including: Call a pre-trained image recognition model to identify the field environment data, and determine the target identification information related to the network element in the field environment data, wherein the type of the network element includes at least one of the following: base station, cell, and the type of the target identification information includes at least one of the following: installation location information and morphology information of the network element.

4. The method according to claim 1, characterized in that Retrieving the target identification information from a business system database includes: Comparing the target identification information with the network element identification information of the corresponding type of each network element stored in the service system database; Determine a network element whose network element identification information is the same as the target identification information, or whose difference satisfies a preset condition, as a target network element that matches the target identification information.

5. The method according to claim 1, characterized in that The map data includes: a geographic information system map, and determining the spatial congestion state from the network element location to the device location based on the map data includes: Determining a vector line segment from the network element location to the device location in the geographic information system map, and determining whether the vector line segment crosses a terrain feature; If the vector line segment passes through a terrain feature, determining that the spatial congestion state is congested; If the vector line segment does not pass through the terrain or objects, it is determined that the spatial congestion state is unblocked.

6. The method according to claim 5, characterized in that The working state of the target network element includes: output signal strength, and predicting the signal state of the device location according to the working state of the target network element and the spatial congestion state includes: In the case where the spatial congestion state is congested, determining the type of terrain and features that the vector line segment passes through, and determining the signal state of the device location according to the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the terrain and features of the corresponding type, and the distance from the network element location to the device location, wherein the signal state includes at least one of the following: the received signal strength at the device location, and the signal coverage state within a preset range around the device location; When the spatial congestion state is unblocked, the signal state of the device location is determined based on the output signal strength of the target network element, the propagation capability information of electromagnetic waves in the air, and the distance from the network element location to the device location.

7. The method according to any one of claims 1 to 6, characterized in that The network element service system is deployed on: a mobile device directly connected to the augmented reality device via a data line, or a cloud server that performs data transmission with the augmented reality device via a wireless network.

8. A multi-source data fusion enhancement device, characterized in that: include: An acquisition module, used to acquire the device position of the augmented reality device and the on-site environment data collected by the augmented reality device, and establish a three-dimensional environment model based on the on-site environment data; an identification module, used to identify target identification information related to a network element in the field environment data, and retrieve the target identification information from a service system database, and if a target network element matching the target identification information is retrieved, obtain the network element location, working status and service data of the target network element; A prediction module, configured to obtain map data from a map database, determine a spatial congestion state from the network element location to the device location based on the map data, and predict a signal state at the device location based on a working state of the target network element and the spatial congestion state; A fusion module, used for fusing the network element position, working status, service data, and signal status of the target network element with the three-dimensional environment model to obtain a comprehensive environment model; A display module is used to send the comprehensive environment model to the augmented reality device for display.

9. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the multi-source data fusion enhancement method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the multi-source data fusion enhancement method according to any one of claims 1 to 7 through the computer program.

Images