Data visualization method and device, electronic equipment and storage medium

Abstract

The invention provides a data visualization method and device, electronic equipment and a storage medium. And the related data of the city is dynamically displayed in real time in a manner of roaming in the panoramic image. For example, a panoramic roaming technology is applied, city scene elements in a city panoramic image currently displayed in a display screen are dynamically matched, and then element data of the city scene elements are displayed in an associated mode in a graphical mode. Therefore, a user can fuse the city scene element currently displayed in the display screen and the element data of the city scene element graphically displayed, so that the user can intuitively and truly feel the general situation data of the city scene element of the panoramic image displayed in the display screen. Through browsing and switching in each city scene element in the panoramic image from the perspective of the first person, different running condition data of the city is represented, so that a user can more easily fuse the city scene element and the element data of the city scene element, and excellent experience is also brought to the user.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a data visualization method, device, electronic device and storage medium. Background Art

[0002] Data visualization is of great significance for the display of urban big data. It can transform huge urban data into intuitive and easy-to-understand images and charts, revealing new trends and opportunities in urban development.

[0003] Through visualization technology, decision makers can quickly capture key issues in urban operations and make scientific and intelligent decisions. Secondly, in the face of emergencies, large-screen visualization of urban big data can gather and analyze relevant urban operation data in real time, provide intuitive and comprehensive information support for emergency command, and improve emergency response efficiency. Large-screen visualization of urban big data is not only a tool for government management, but also a window for citizens to understand the city and participate in urban governance, enhancing citizens' sense of belonging and participation. Summary of the invention

[0004] The present application illustrates a data visualization method, device, electronic device and storage medium.

[0005] In a first aspect, the present application provides a data visualization method, the method comprising:

[0006] Obtain a panoramic image corresponding to the real scene of a target area in a target city;

[0007] Loading a panoramic image corresponding to the real scene of the target area in the target city into the rendering engine;

[0008] Displaying a partial area of ​​the panoramic image loaded in the rendering engine on the display screen;

[0009] During roaming in the panoramic image, detecting target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen;

[0010] Searching for element data of a target city scene element among element data of a plurality of city scene elements set in advance;

[0011] Searching for a target graphical component suitable for element data of a target city scene element among multiple types of graphical components set in advance, wherein the multiple types of graphical components set in advance at least include: an icon component, an image component, a table component, a label component, a chart component, and a video component;

[0012] Based on the target graphical component, the element data of the target city scene element is displayed in the area corresponding to the target city scene element on the display screen.

[0013] In a second aspect, the present application provides a data visualization device, the device comprising:

[0014] An acquisition module, used to acquire a panoramic image corresponding to a real scene of a target area in a target city;

[0015] A loading module, used for loading a panoramic image corresponding to a real scene of a target area in a target city into a rendering engine;

[0016] A first display module, used for displaying a partial area of ​​the panoramic image loaded in the rendering engine on a display screen;

[0017] A detection module, used to detect target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen during roaming in the panoramic image;

[0018] A first search module, used to search for element data of a target city scene element among element data of a plurality of city scene elements set in advance;

[0019] A second search module is used to search for a target graphical component suitable for element data of a target city scene element among a plurality of types of graphical components set in advance, wherein the plurality of types of graphical components set in advance at least include: an icon component, an image component, a table component, a label component, a chart component, and a video component;

[0020] The second display module is used to display the element data of the target city scene element in the area corresponding to the target city scene element on the display screen based on the target graphical component.

[0021] In a third aspect, the present application shows an electronic device, which includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in any of the above aspects.

[0022] In a fourth aspect, the present application illustrates a non-temporary computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute a method as described in any of the above aspects.

[0023] In a fifth aspect, the present application illustrates a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any of the above aspects.

[0024] The technical solution provided by this application may have the following beneficial effects:

[0025] This application uses a roaming method in a panoramic image to dynamically display relevant data of the city (such as relevant data on the operation of the city, etc.) in real time. For example, by using the panoramic roaming technology, the city scene elements in the panoramic image of the city currently displayed on the display screen are dynamically matched, and then the element data of the city scene elements are associated and displayed in a graphical manner, so that the user can combine the element data of the city scene elements currently displayed on the display screen and the city scene elements displayed graphically for viewing.

[0026] The panoramic image is used to fully display the real physical appearance of the city, and roaming in the panoramic image is supported. During the roaming process in the panoramic image, the scene of the city will be dynamically changed. For example, the city scene elements in the panoramic image of the city are dynamically changed in the display screen, and then dynamically displayed in the display screen: the element data of the city scene elements displayed in the display screen can allow users to intuitively and truly feel the overview data of the city scene elements of the panoramic image displayed in the display screen, feel the real physical world and its associated data, and browse and switch between the various city scene elements in the panoramic image from a first-person perspective to show the different operating data of the city, so that users can more easily integrate the city scene elements with the element data of the city scene elements, and also bring users an excellent experience. Secondly, in this application, the displayed area and data are no longer restricted by the size of the display screen. By displaying part of the area in the panoramic image on the display screen and roaming in the panoramic image, a borderless visualization effect can be achieved. In addition, there is no need to build a 3D model of the city, so the cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a step flow chart of a data visualization method of the present application.

[0028] Figure 2 It is a structural block diagram of a data visualization device of the present application.

[0029] Figure 3 It is a block diagram of an electronic device of the present application.

[0030] Figure 4 It is a block diagram of an electronic device of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] Urban big data has the following characteristics: it uses urban scenes as the spatial base to extend various data on urban operations. The data volume is large, the data types are diverse, and the forms of expression required are also diverse.

[0033] Secondly, urban big data needs to have a macro-level presentation, but also needs to go deep into each detailed scene, so there are many levels and changes, and it needs to be presented to users in an all-round way.

[0034] The visualization of urban big data is a technical means to present the huge and complex data sets of urban operation through visualization tools such as graphics, charts, maps, and dashboards. By analyzing and processing the massive data in the city, the data is presented in an intuitive and easy-to-understand form, so that people can intuitively understand and analyze the data, providing new perspectives and references for urban planning, construction, and management. This process is not limited to the simple display of data, but more importantly, it reveals the patterns, trends, and associations behind the data in a visual way, thereby helping users quickly and accurately obtain relevant data of the city and make scientific decisions based on the relevant data of the city.

[0035] A single common large-screen visualization interface displays the city's big data. Due to the limitation of the size of the large-screen interface, it can only display the city's big data that the large-screen interface can cover, and the effect is not ideal.

[0036] To this end, the solution of this application is proposed.

[0037] For example, a method of using panoramic roaming is proposed to dynamically display relevant data of the city (such as city operation-related data, etc.) in real time. For example, using panoramic roaming technology, the city scene elements in the panoramic image of the city displayed on the display screen are dynamically matched, and then the element data of the city scene elements are displayed in association, so that the user can integrate the city scene elements and the element data of the city scene elements for viewing.

[0038] Compared with other methods, panoramic images are more three-dimensional and can better highlight the appearance of specific objects. Panoramic images rely on taking pictures of the real physical world to more realistically show the real physical world. At the same time, panoramic images are from a first-person perspective, creating an effect of roaming in a world, allowing the observer's field of vision to continuously change progressively. Compared with 3D city modeling, panoramic images are lighter, simpler to produce, lower cost, shorter production cycle, and easier to use.

[0039] In this way, by building a visual display of urban data through panoramic images, we can take into account both speed and cost. While achieving the purpose and effect of the expected expression of the data, we can have a faster construction speed and a more economical output ratio.

[0040] Before introducing the solution of this application, the technical terms that may be involved in this application are first explained.

[0041] Panoramic image: A panoramic image is a view of a 3D (3-Dimensional) real scene. Panoramic image is a view based on the emerging rich media technology - 3D real scene. It simulates a person's full-view in a 3D model, giving people an immersive and realistic feeling. Panoramic images are divided into image panoramas and video panoramas. According to interactivity, they can be divided into non-interactive information panoramas and interactive information panoramas. Panoramic images refer to those that can provide free browsing of 360 degrees in azimuth in the horizontal direction and 180 degrees in the vertical direction from a fixed observation point. Simplified panoramas can only provide 360 ​​degrees browsing in the horizontal direction.

[0042] Big data: mainly refers to urban big data, which refers to the data generated or obtained during the operation of the city and its related activity elements.

[0043] Borderless: A panoramic field of view generated by panoramic images. Compared with the traditional visualization interface, the visible area will be infinitely expanded and will not be restricted or locked by the device window size (display screen size).

[0044] SIFT, Scale Invariant Feature Transform, is a classic computer vision algorithm used to extract feature points from images. Its core purpose is to extract some key points that remain unchanged under changes such as image scaling and rotation, so that images can be matched through these key points. These key points are usually local features in the image, such as corners and edges. The task of SIFT is to find these features and describe them so that they can be used for comparison between different images.

[0045] Nginx, engine x, is a high-performance HTTP (HyperText Transfer Protocol) and reverse proxy web server.

[0046] Reference Figure 1 , shows a flow chart of the steps of a data visualization method of the present application, wherein the method comprises:

[0047] In step S101, a panoramic image corresponding to a real scene of a target area in a target city is obtained.

[0048] The target area in the target city may be a part of the target city, or an area designated by a technician, etc. For example, an area of ​​a park in the city, an area of ​​an industrial park in the city, an area of ​​a university campus in the city, and the like.

[0049] The panoramic image corresponding to the real scene of the target area in the target city may be pre-set and stored in the electronic device or in the cloud. In this way, the panoramic image corresponding to the real scene of the target area in the target city stored in advance in the electronic device or in the cloud may be obtained.

[0050] The process of presetting the panoramic image corresponding to the real scene of the target area in the target city may include:

[0051] 11) Real scene shooting of the target area of ​​the target city is performed to obtain multiple real scene images of the target area of ​​the target city.

[0052] For example, a drone equipped with professional camera equipment can be used to take low-altitude photos of the target area in the target city to obtain real-life data of the target area in the target city.

[0053] The professional camera device may be a camera that supports panoramic photography.

[0054] 12) Synthesize multiple real-scene images of the target area of ​​the target city to obtain a complete panoramic image.

[0055] The multiple real-life images taken by drones in the target area of ​​the target city are often images from different positions and different perspectives. Therefore, there may be overlapping areas between the multiple real-life images. In this way, the multiple real-life images of the target area of ​​the target city can be synthesized with the help of feature extraction (such as SIFT) and matching, feature matching optimization, image transformation structure calculation, image mapping, image registration and image fusion methods to obtain a complete panoramic image, that is, a large-scale panoramic image.

[0056] 13) Crop the redundant borders and unused areas of the obtained complete panoramic image, and further adjust the exposure, contrast, color and other parameters of the complete panoramic image according to the instructions of the technician.

[0057] The extra boundaries may be those specified by the technician in the complete panoramic image, etc.

[0058] 14) Using panoramic image projection technology, the panoramic image outputted from step 13) is converted into a panoramic image for representation on a two-dimensional plane.

[0059] Panoramic image projection technology is a technology that converts 360-degree images or videos into two-dimensional plane representations. The main purpose of panoramic image projection is to present an infinite visual experience on a display screen of limited size, allowing users to feel immersive.

[0060] The panoramic image projection types include: spherical projection, cylindrical projection, fisheye projection, etc. In one embodiment, spherical projection, etc. can be selected. Spherical projection can include: Equirectangular (equidistant cylindrical projection) and Cubemap (cubemap), etc.

[0061] 15) The panoramic image outputted in step 14) for two-dimensional plane representation is stored in a resource directory, and then published using a static resource publishing middleware (such as Nginx) to facilitate subsequent calls when the panoramic image outputted in step 14) for two-dimensional plane representation is needed.

[0062] For example, the panoramic image outputted in step 14) for two-dimensional plane representation is stored according to the name of the target city and the area identifier of the target area (such as the coordinate range of the area, the number of the area or the name of the unit where the area is located, etc.) as an index.

[0063] Different cities have different names, and different areas in the same city have different area identifiers.

[0064] Thus, when obtaining the panoramic image corresponding to the real scene of the target area in the target city in step S101, the electronic device can call the panoramic image corresponding to the real scene of the target area in the target city through the static resource publishing middleware, for example, the electronic device submits the name of the target city and the area identifier of the target area to the static resource publishing middleware, so that the static resource publishing middleware uses the name of the target city and the area identifier of the target area as indexes to search for the stored panoramic image corresponding to the real scene of the target area in the target city, and returns the panoramic image corresponding to the real scene of the target area in the target city. Thus, the electronic device obtains the panoramic image corresponding to the real scene of the target area in the target city.

[0065] In step S102, a panoramic image corresponding to a real scene of a target area in a target city is loaded into a rendering engine.

[0066] The rendering engine in this application supports the rendering of panoramic images. This application does not limit the specific rendering engine, as long as it can render panoramic images.

[0067] In one example, rendering engines may include: V-Ray (a high-quality rendering software produced by chaosgroup and asgvis), Corona Renderer (a modern high-performance (un)biased photorealistic renderer), and Octane Render (a renderer belonging to Efractive Software, used in Windows systems), etc.

[0068] In one embodiment of the present application, the panoramic image outputted in step 14) for representation in a two-dimensional plane is large in size and occupies a large space. Thus, loading the panoramic image outputted in step 14) for representation in a two-dimensional plane in the rendering engine often takes a long time and has low loading efficiency.

[0069] To this end, in another embodiment of the present application, when the panoramic image for representation in a two-dimensional plane outputted from step 14) is stored according to the resource directory in 15), the panoramic image for representation in a two-dimensional plane outputted from step 14) may be cut into a plurality of panoramic sub-images, and each panoramic sub-image may be saved according to its position in the panoramic image, for example, the correspondence between each panoramic sub-image and its position in the panoramic image may be saved.

[0070] In this way, when obtaining the panoramic image corresponding to the real scene of the target area in the target city in step S101, the electronic device can obtain multiple panoramic sub-images obtained by cutting the panoramic image corresponding to the real scene of the target area in the target city in advance and the positions of each panoramic sub-image in the panoramic image.

[0071] For example, the electronic device can call the panoramic image corresponding to the real scene of the target area in the target city through the static resource publishing middleware. For example, the electronic device submits the name of the target city and the area identifier of the target area to the static resource publishing middleware, so that the static resource publishing middleware uses the name of the target city and the area identifier of the target area as indexes to search for the stored panoramic image corresponding to the real scene of the target area in the target city, and the multiple panoramic sub-images obtained by cutting and the positions of each panoramic sub-image in the panoramic image, and returns the multiple panoramic sub-images and the positions of each panoramic sub-image in the panoramic image. So that the electronic device can obtain the multiple panoramic sub-images and the positions of each panoramic sub-image in the panoramic image.

[0072] Accordingly, when the panoramic image corresponding to the real scene of the target area in the target city is loaded in the rendering engine in step S102, each panoramic sub-image can be loaded in parallel in the rendering engine according to the position of each panoramic sub-image in the panoramic image, thereby realizing asynchronous loading / parallel loading of multiple panoramic sub-images, improving loading efficiency and saving time.

[0073] In step S103, a partial area of ​​the panoramic image loaded in the rendering engine is displayed on the display screen.

[0074] In the present application, the size of the display screen is limited, while the size of the panoramic image is large. The display screen may usually not be able to display all the contents of the panoramic image at the same time. Part of the contents of the panoramic image can be displayed on the display screen at any time.

[0075] When the panoramic image is just being displayed on the display screen, a default area in the panoramic image may be displayed on the display screen. The default area is a partial area in the panoramic image. The default area may be an area designated in advance by a technician, and the like.

[0076] The user can subsequently roam in the panoramic image by manipulating the mouse or inputting touch operations on the display screen. Roaming can be understood as changing the position and / or viewing angle in the panoramic image, so as to switch to display other areas in the panoramic image on the display screen.

[0077] Alternatively, the electronic device may automatically roam through the panoramic image.

[0078] For example, in the present application, the electronic device also includes a roaming controller and a scene controller.

[0079] The roaming controller is used to arrange the switching order of various city scene elements on the display screen during roaming in the panoramic image, to arrange the roaming path during roaming in the panoramic image, to arrange the roaming speed during roaming in the panoramic image, etc.

[0080] The technicians can arrange in advance according to the roaming controller the switching order of various city scene elements on the display screen during roaming in the panoramic image, the roaming path during roaming in the panoramic image, and the roaming speed during roaming in the panoramic image, so that the electronic device can automatically roam in the panoramic image according to the technicians' prior arrangement.

[0081] The scene controller is used to display the city scene elements and element data of the city scene elements in the panoramic image on the display screen, schedule the switching of the city scene elements according to the user's instructions, and control various translations and zooms of the city scene elements.

[0082] In step S104, during the roaming in the panoramic image, target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen are detected.

[0083] In one embodiment of the present application, target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen may be detected based on a template matching method.

[0084] The image templates of each urban scene element in the target city can be set in advance. Thus, in this step, for the image template of any urban scene element in the target city, based on the image template, a part identical or similar to the image template can be searched in a partial area of ​​the panoramic image currently displayed on the display screen, wherein the partial area of ​​the panoramic image currently displayed on the display screen can be first converted into a grayscale image, and then the part identical or similar to the image template can be searched in the grayscale image, and then the elements in the identical or similar part are used as the target urban scene element, that is, the template matching is successful. The same is true for the image templates of each other urban scene element in the target city.

[0085] A matching threshold may be set in advance, and the specific value of the matching threshold may be determined according to actual conditions, and this application does not impose any limitation on this.

[0086] If the similarity between a certain part in the partial area of ​​the panoramic image currently displayed on the display screen and the image template is greater than the matching threshold, it means that the part is the same as or similar to the image template.

[0087] The template matching process may adopt a correlation method, whose formula is usually based on the product sum of pixel values ​​to evaluate the similarity between the image template and the image to be matched.

[0088] In one example, the correlation formula may be a Normalized Cross-Correlation (NCC) coefficient, which measures the linear correlation between two image regions.

[0089]

[0090] T(x,y) is the pixel value of the image template at position (x,y).

[0091] I(x,y) is the pixel value of the image to be matched at the corresponding position (x,y) (here it is assumed that the image template and the image to be matched have the same size in the area currently considered).

[0092] is the average value of all pixel values ​​of the image template, calculated as Where N is the total number of pixels in the template.

[0093] is the average value of all pixel values ​​in the current considered area (same size as the template) of the image to be matched, calculated as

[0094] Summation operation Σ x，y Traverse all pixel positions of the image template (or the currently considered area of ​​the image to be matched).

[0095] In step S105, the element data of the target city scene element is searched for in the element data of a plurality of city scene elements that are set in advance.

[0096] The urban scene elements in the target area in the target city may include: entities in the target area in the target city, including: buildings, flowers, trees, rivers, signs, roads, gates, shops, scenic spots, bus stations, subway stations, rental stations, passenger stations, overpasses, pedestrian overpasses, parking lots, roadside parking spaces, intersections, traffic lights, road signs, traffic rules signs, fire hydrants, etc.

[0097] The element data of the urban scene elements include: a general description of the urban scene elements, an introduction, the time they have been put into use, the expiration date, the objects of service, how to use them, precautions for use, and the current status, etc.

[0098] In the present application, for any urban scene element in the target area of ​​the target city, the technicians can set (arrange, write, summarize and count, etc.) the element data of the urban scene element in advance, and then form a corresponding table item with the element identifier of the urban scene element and the element data of the urban scene element, and store it in the first corresponding relationship between the element identifier of the urban scene element and the element data of the urban scene element set in advance.

[0099] Different city scene elements have different element identifiers. For example, different city scene elements in the same city have different element identifiers. The element identifier of a city scene element may be an ID (Identity Document) of the city scene element, and the like.

[0100] The same is true for every other urban scene element in the target area in the target city.

[0101] In this way, in this step, the element data corresponding to the element identifier of the target city scene element can be found in the first correspondence between the element identifier of the city scene element and the element data of the city scene element that is set in advance, and used as the element data of the target city scene element.

[0102] In step S106, a target graphical component suitable for the element data of the target city scene element is searched among the multiple types of graphical components set in advance.

[0103] The various types of graphical components that are set in advance include at least: icon components, image components, table components, label components, chart components and video components.

[0104] The graphical component may be an SDK (Software Development Kit) or the like.

[0105] Various types of graphical components are developed in advance by technicians, and various types of graphical components can be called and used by electronic devices. For example, various types of graphical components have their own component identifiers, and the component identifiers of the graphical components may include the API (Application Programming Interface) of the graphical components. Different types of graphical components have different component identifiers. For example, different types of graphical components have different APIs. Electronic devices can call graphical components through the component identifiers (such as APIs) of the graphical components.

[0106] The graphical component has processing logic for graphically processing the element data of the city scene elements. The processing logic for graphically processing the element data of the city scene elements in different types of graphical components is different.

[0107] Icon components, image components, table components, label components, chart components and video components are different types of graphical components. Chart components include: pie chart components, bar chart components and line chart components.

[0108] In the present application, for the element data of any urban scene element in the target area of ​​the target city, the technicians can set up a graphical component applicable to the element data of the urban scene element in advance. The graphical component applicable to the element data of the urban scene element can be understood as: after graphically processing the element data of the urban scene element using the graphical component, the element data of the urban scene element can be displayed more clearly and comprehensively, making it easier for users to understand the element data of the urban scene element, and users can more easily perceive the key points of the element data of the urban scene element.

[0109] Then, the element data of the city scene element and the component identification of the graphical component applicable to the element data of the city scene element are formed into corresponding table items and stored in a second corresponding relationship set in advance between the element data of the city scene element and the component identification of the graphical component applicable to the element data of the city scene element.

[0110] The same is true for the element data of each other urban scene element in the target area in the target city.

[0111] Thus, in this step, the component identification of the graphical component corresponding to the element data of the target city scene element can be searched in the pre-set second correspondence between the element data of the city scene element and the component identification of the graphical component applicable to the element data of the city scene element; the graphical component corresponding to the component identification is called and used as the target graphical component applicable to the element data of the target city scene element.

[0112] In step S107, based on the target graphical component, the element data of the target city scene element is displayed in the area corresponding to the target city scene element on the display screen.

[0113] The area corresponding to the target city scene element includes: the interior of the area where the target city scene element is located or the outer periphery of the area where the target city scene element is located (for example, the periphery of the area or the vicinity of the area, etc.).

[0114] In one embodiment of the present application, when the element data of the target city scene element is displayed in the area corresponding to the target city scene element on the display screen based on the target graphical component, the pre-set reference position of the target graphical component in the panoramic image can be obtained.

[0115] The reference position of the target graphical component in the panoramic image may be set in advance by a technician.

[0116] The reference position may be located inside the area where the target city scene element in the panoramic image is located, or the reference position may be located outside the area where the target city scene element in the panoramic image is located (eg, around the area or near the area, etc.).

[0117] Then, the target display position of the target graphical component on the display screen may be determined according to the reference position.

[0118] For example, the reference position is searched for in a partial area of ​​the panoramic image displayed on the display screen and is used as the target display position.

[0119] Afterwards, the element data of the target city scene element may be displayed at the target display position on the display screen.

[0120] In the present application, for any city scene element in the target area of ​​the target city, the technician can set in advance the reference position of the graphical component of the element data applicable to the city scene element in the panoramic image. The reference position can be located inside the area where the city scene element is located in the panoramic image, or the reference position can also be located around the area where the city scene element is located in the panoramic image. The specific position of the reference position inside the area where the city scene element is located can be set by the technician according to actual conditions, or the specific position of the reference position around the area where the city scene element is located can be set by the technician according to actual conditions. Then, the element data of the city scene element and the reference position of the graphical component of the element data applicable to the city scene element in the panoramic image can be formed into a corresponding table item, and stored in the third corresponding relationship between the element data of the city scene element and the reference position of the graphical component of the element data applicable to the city scene element in the panoramic image, which is set in advance.

[0121] The same is true for every other urban scene element in the target area in the target city.

[0122] In this way, when obtaining the pre-set reference position of the target graphical component in the panoramic image, the reference position corresponding to the element data of the target city scene element can be found in the third correspondence between the pre-set element data of the city scene element and the reference position of the graphical component applicable to the element data of the city scene element in the panoramic image, and used as the reference position of the target graphical component in the panoramic image.

[0123] In another embodiment of the present application, when the element data of the target city scene element is displayed in the area corresponding to the target city scene element on the display screen based on the target graphical component, the central area of ​​the target city scene element displayed on the display screen can be determined. The central area of ​​the target city scene element can be the geometric center of the target city scene element, etc. The element data of the target city scene element is displayed in the central area of ​​the target city scene element displayed on the display screen, so that the user can intuitively feel that the displayed element data is for the target city scene element, rather than for other city scene elements, and the overlap of the graphical components corresponding to different city scene elements displayed on the display screen can be avoided as much as possible, so as to avoid affecting the user's viewing of the content in the graphical components corresponding to different city scene elements.

[0124] Among them, displaying the element data of urban scene elements on the display screen based on the graphical component can be understood as: the graphical component graphically processes the element data of the urban scene elements through its internal processing logic for graphically processing the element data of the urban scene elements, and then renders the graphical component on the display screen. The rendered graphical component has relevant content of the graphically processed element data, which is equivalent to displaying the element data of the urban scene elements in a graphical manner, so that users can view the relevant content of the graphically processed element data, so that users can view the element data of the urban scene elements in a graphical manner.

[0125] This application uses a roaming method in a panoramic image to dynamically display relevant data of the city (such as relevant data on the operation of the city, etc.) in real time. For example, by using the panoramic roaming technology, the city scene elements in the panoramic image of the city currently displayed on the display screen are dynamically matched, and then the element data of the city scene elements are associated and displayed in a graphical manner, so that the user can combine the element data of the city scene elements currently displayed on the display screen and the city scene elements displayed graphically for viewing.

[0126] The panoramic image is used to fully display the real physical appearance of the city, and roaming in the panoramic image is supported. During the roaming process in the panoramic image, the scene of the city will be dynamically changed. For example, the city scene elements in the panoramic image of the city are dynamically changed in the display screen, and then dynamically displayed in the display screen: the element data of the city scene elements displayed in the display screen can allow users to intuitively and truly feel the overview data of the city scene elements of the panoramic image displayed in the display screen, feel the real physical world and its associated data, and browse and switch between the various city scene elements in the panoramic image from a first-person perspective to show the different operating data of the city, so that users can more easily integrate the city scene elements with the element data of the city scene elements, and also bring users an excellent experience. Secondly, in this application, the displayed area and data are no longer restricted by the size of the display screen. By displaying part of the area in the panoramic image on the display screen and roaming in the panoramic image, a borderless visualization effect can be achieved. In addition, there is no need to build a 3D model of the city, so the cost is low.

[0127] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the present application.

[0128] Reference Figure 2 , showing a data visualization device of the present application, the device comprising:

[0129] An acquisition module 11 is used to acquire a panoramic image corresponding to a real scene of a target area in a target city;

[0130] A loading module 12, used for loading a panoramic image corresponding to a real scene of a target area in a target city into a rendering engine;

[0131] A first display module 13, used for displaying a partial area of ​​the panoramic image loaded in the rendering engine on a display screen;

[0132] A detection module 14, used to detect target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen during roaming in the panoramic image;

[0133] A first search module 15 is used to search for element data of a target city scene element in element data of a plurality of city scene elements that are set in advance;

[0134] A second search module 16 is used to search for a target graphical component suitable for element data of a target city scene element among multiple types of graphical components set in advance, wherein the multiple types of graphical components set in advance at least include: an icon component, an image component, a table component, a label component, a chart component, and a video component;

[0135] The second display module 17 is used to display the element data of the target city scene element in the area corresponding to the target city scene element on the display screen based on the target graphical component.

[0136] In an optional implementation, the acquisition module includes:

[0137] A first acquisition unit is used to acquire a plurality of panoramic sub-images obtained by cutting a panoramic image corresponding to a real scene of a target area in a target city in advance, and positions of the respective panoramic sub-images in the panoramic image;

[0138] Accordingly, the loading module includes:

[0139] The loading unit is used to load the panoramic sub-images in parallel in the rendering engine according to the positions of the panoramic sub-images in the panoramic image.

[0140] In an optional implementation, the first search module includes:

[0141] The first search unit is used to search for element data corresponding to the element identifier of the target urban scene element in a pre-set first correspondence between the element identifier of the urban scene element and the element data of the urban scene element, and use the element data as the element data of the target urban scene element.

[0142] In an optional implementation, the second search module includes:

[0143] A second searching unit is used to search for a component identification of a graphical component corresponding to the element data of a target urban scene element in a pre-set second correspondence between the element data of the urban scene element and the component identification of the graphical component applicable to the element data of the urban scene element;

[0144] The calling unit is used to call the graphical component corresponding to the component identifier and use it as a target graphical component suitable for element data of a target city scene element.

[0145] In an optional implementation, the second display module includes:

[0146] A second acquisition unit is used to acquire a pre-set reference position of the target graphical component in the panoramic image; the reference position is located inside the area where the target city scene element in the panoramic image is located, or the reference position is located outside the area where the target city scene element in the panoramic image is located;

[0147] A first determining unit, configured to determine a target display position of a target graphical component on a display screen according to a reference position;

[0148] The first display unit is used to display element data of a target city scene element at a target display position on a display screen.

[0149] In an optional implementation, the second acquiring unit includes:

[0150] A search subunit is used to search for a reference position corresponding to the element data of a target urban scene element in a third correspondence relationship set in advance between the element data of the urban scene element and the reference position of a graphical component applicable to the element data of the urban scene element in the panoramic image, and use the reference position as the reference position of the target graphical component in the panoramic image.

[0151] In an optional implementation, the second display module includes:

[0152] A second determining unit, used to determine a central area of ​​a target city scene element displayed on the display screen;

[0153] The second display unit is used to display element data of the target city scene element in the central area of ​​the target city scene element displayed on the display screen.

[0154] This application uses a roaming method in a panoramic image to dynamically display relevant data of the city (such as relevant data on the operation of the city, etc.) in real time. For example, by using the panoramic roaming technology, the city scene elements in the panoramic image of the city currently displayed on the display screen are dynamically matched, and then the element data of the city scene elements are associated and displayed in a graphical manner, so that the user can combine the element data of the city scene elements currently displayed on the display screen and the city scene elements displayed graphically for viewing.

[0155] The panoramic image is used to fully display the real physical appearance of the city, and roaming in the panoramic image is supported. During the roaming process in the panoramic image, the scene of the city will be dynamically changed. For example, the city scene elements in the panoramic image of the city are dynamically changed in the display screen, and then dynamically displayed in the display screen: the element data of the city scene elements displayed in the display screen can allow users to intuitively and truly feel the overview data of the city scene elements of the panoramic image displayed in the display screen, feel the real physical world and its associated data, and browse and switch between the various city scene elements in the panoramic image from a first-person perspective to show the different operating data of the city, so that users can more easily integrate the city scene elements with the element data of the city scene elements, and also bring users an excellent experience. Secondly, in this application, the displayed area and data are no longer restricted by the size of the display screen. By displaying part of the area in the panoramic image on the display screen and roaming in the panoramic image, a borderless visualization effect can be achieved. In addition, there is no need to build a 3D model of the city, so the cost is low.

[0156] Optionally, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0157] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0158] Figure 3 800 is a block diagram of an electronic device 800 shown in the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0159] Reference Figure 3 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0160] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0161] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0162] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0163] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also monitor the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0164] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0165] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0166] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can monitor the open / closed state of the device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also monitor the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to monitor the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0167] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast operation information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0168] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0169] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0170] Figure 4 1 is a block diagram of an electronic device 1900 shown in the present application. For example, the electronic device 1900 may be provided as a server.

[0171] Reference Figure 4 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0172] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0173] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0175] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0176] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0177] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0178] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0182] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data visualization method, characterized in that: The method comprises: Obtain a panoramic image corresponding to the real scene of a target area in a target city; Loading a panoramic image corresponding to the real scene of the target area in the target city into the rendering engine; Displaying a partial area of ​​the panoramic image loaded in the rendering engine on the display screen; During roaming in the panoramic image, detecting target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen; Searching for element data of a target city scene element among element data of a plurality of city scene elements set in advance; Searching for a target graphical component suitable for element data of a target city scene element among multiple types of graphical components set in advance, wherein the multiple types of graphical components set in advance at least include: an icon component, an image component, a table component, a label component, a chart component, and a video component; Based on the target graphical component, the element data of the target city scene element is displayed in the area corresponding to the target city scene element on the display screen.

2. The method according to claim 1, characterized in that The step of obtaining a panoramic image corresponding to a real scene of a target area in a target city includes: Acquire multiple panoramic sub-images obtained by cutting the panoramic image corresponding to the real scene of the target area in the target city in advance, and the positions of the respective panoramic sub-images in the panoramic image; Accordingly, the step of loading the panoramic image corresponding to the real scene of the target area in the target city in the rendering engine includes: The panoramic sub-images are loaded in parallel in the rendering engine according to their respective positions in the panoramic image.

3. The method according to claim 1, characterized in that The step of searching the element data of the target city scene element from the element data of the plurality of city scene elements set in advance includes: In the first correspondence relationship between the element identifier of the urban scene element and the element data of the urban scene element set in advance, the element data corresponding to the element identifier of the target urban scene element is searched and used as the element data of the target urban scene element.

4. The method according to claim 1, characterized in that: The step of searching for a target graphical component suitable for element data of a target city scene element among the plurality of types of graphical components set in advance includes: In a pre-set second correspondence between element data of the city scene element and component identifiers of the graphical components applicable to the element data of the city scene element, searching for the component identifier of the graphical component corresponding to the element data of the target city scene element; The graphical component corresponding to the component identifier is called and used as the target graphical component for the element data applicable to the target city scene element.

5. The method according to claim 1, characterized in that The target-based graphical component displays the element data of the target city scene element in the area corresponding to the target city scene element on the display screen, including: Acquire a pre-set reference position of a target graphical component in the panoramic image; the reference position is located inside a region where a target city scene element in the panoramic image is located, or the reference position is located outside a region where a target city scene element in the panoramic image is located; Determining a target display position of a target graphical component on a display screen according to the reference position; The element data of the target city scene element is displayed at the target display position on the display screen.

6. The method according to claim 5, characterized in that The obtaining of a preset reference position of a target graphical component in the panoramic image includes: In the third correspondence relationship set in advance between the element data of the urban scene element and the reference position of the graphical component applicable to the element data of the urban scene element in the panoramic image, the reference position corresponding to the element data of the target urban scene element is found and used as the reference position of the target graphical component in the panoramic image.

7. The method according to claim 1, characterized in that The target-based graphical component displays the element data of the target city scene element in the area corresponding to the target city scene element on the display screen, including: Determine a central area of ​​a target city scene element displayed on a display screen; The element data of the target city scene element is displayed in the central area of ​​the target city scene element displayed on the display screen.

8. A data visualization device, characterized in that: The device comprises: An acquisition module, used to acquire a panoramic image corresponding to a real scene of a target area in a target city; A loading module, used for loading a panoramic image corresponding to a real scene of a target area in a target city into a rendering engine; A first display module, used for displaying a partial area of ​​the panoramic image loaded in the rendering engine on a display screen; A detection module, used to detect target city scene elements in a partial area of ​​the panoramic image currently displayed on the display screen during roaming in the panoramic image; A first search module, used to search for element data of a target city scene element among element data of a plurality of city scene elements set in advance; A second search module is used to search for a target graphical component suitable for element data of a target city scene element among a plurality of types of graphical components set in advance, wherein the plurality of types of graphical components set in advance at least include: an icon component, an image component, a table component, a label component, a chart component, and a video component; The second display module is used to display the element data of the target city scene element in the area corresponding to the target city scene element on the display screen based on the target graphical component.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images