Intelligent data display method and system integrating image processing and multi-dimensional visualization

Abstract

The invention discloses an intelligent data display method and system fusing image processing and multi-dimensional visualization. The method comprises the following steps: acquiring a plurality of to-be-processed original images and structured data from different sources; preprocessing the plurality of to-be-processed original images; performing enhancement processing on the cleaned image; evaluating the image quality, and verifying; performing pattern recognition on the qualified image, extracting key features of the image, and performing quantification to obtain a data model; performing association analysis on the data model and the structured data to form a data wide table; performing multi-dimensional data visualization analysis on the data wide table, drawing a visualization area graph and designing interaction; and displaying the data on the interactive user interface. According to the invention, by adopting an image data intelligent processing and analysis process, effective processing and display of a large amount of image data are realized, the processing normalization and analysis quality of the image data are improved, and the availability and interpretability of the image data are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of data visualization, and specifically relates to an intelligent data display method and system integrating image processing and multi-dimensional visualization. Background Art

[0002] With the rapid development and progress of Internet of Things technology, a large number of sensors are installed in production equipment, which continuously generate various types of real-time data. The widespread use of equipment such as drones and surveillance cameras has also led to a sharp increase in the amount of image data. This information includes both structured data and semi-structured data. Semi-structured data often has non-standard formats, high dimensions, and complexity, making it extremely challenging to extract valuable information from it.

[0003] Traditional visual data processing methods usually focus on the analysis of structured data, and have weak processing capabilities for semi-structured data such as images. Existing image processing technologies focus on various types of image analysis and processing, but lack the integration and processing of multi-source image data, as well as multi-dimensional visualization, which limits the depth and breadth of data analysis.

[0004] How to effectively extract key information from large amounts of semi-structured data, transform it into actionable knowledge, and aggregate it into a panoramic view to support decision makers has become an urgent problem to be solved.

[0005] Therefore, the present invention aims to propose an intelligent data display method and system that integrates image processing and multi-dimensional visualization, and is committed to extracting key information from concrete, low-value images, analyzing and associating them with structured information, and transforming them into abstract, high-value data models, and then presenting them in an intuitive and easy-to-understand visualization. In this way, valuable information can be extracted from massive image data, breaking down the analytical barriers between structured and unstructured data, and providing strong support for activity decision-making. Summary of the invention

[0006] The purpose of the present invention is to provide an intelligent data display method and system that integrates image processing and multi-dimensional visualization to solve the problem that a large amount of image data cannot be efficiently processed, analyzed and displayed.

[0007] The present invention is achieved through the following technical solutions:

[0008] An intelligent data display method integrating image processing and multi-dimensional visualization, comprising:

[0009] S1. Obtain multiple raw images and structured data to be processed from different sources;

[0010] S2, cleaning the multiple original images;

[0011] S3, performing enhancement processing on the cleaned image to improve the image quality;

[0012] S4. Verify image quality to ensure that the clarity, saturation and information content of the processed image meet the quality requirements;

[0013] S5. Perform pattern recognition on the images that have passed the quality inspection, extract key features of the images and quantify them to obtain a data model;

[0014] S6. Perform correlation analysis on the data model and structured data to form a wide data table;

[0015] S7. Perform multidimensional data visualization analysis on wide data tables, draw visualization area graphics and design interactions;

[0016] S8. Display data in an interactive user interface.

[0017] Furthermore, in some embodiments, the step of performing enhancement processing on the cleaned image in S3 includes:

[0018] S31, increasing or decreasing the overall brightness of the image to eliminate the image brightness inconsistency caused by illumination differences;

[0019] S32, adjusting the pixel value distribution of the image by equalization to enhance the contrast;

[0020] S33, enhancing the details and edges of the image and performing sharpening processing;

[0021] S34. Adjust the image resolution to a standard value. For low-resolution images, use a generative adversarial network to perform super-resolution reconstruction on the image.

[0022] Furthermore, in some embodiments, the step of using a generative adversarial network to perform super-resolution reconstruction on the image in step S34 includes:

[0023] S341, constructing a training set and a validation set, including a high-resolution image and its corresponding low-resolution image;

[0024] S342, designing a neural network structure of a generator and defining a loss function, wherein the generator is used to generate a high-resolution image from a low-resolution image, with the generated image being judged as true as a training target;

[0025] S434, designing a neural network structure of a discriminator and defining a loss function, wherein the discriminator is used to distinguish between real images and generated images, and the generated images are judged as false as a training target;

[0026] S345, initializing the weights of the generator and the discriminator to train the model, and alternately updating the weights of the two during the training process;

[0027] S346, verify the model effect, apply the model to the image processed in step S3, and standardize the image resolution.

[0028] Furthermore, in some embodiments, the step S5 performs pattern recognition on the image that has passed the quality verification, extracts key features of the image and quantifies them to obtain a data model, including

[0029] S51, extracting color, edge, and corner features from the preprocessed image;

[0030] S52, cascading the obtained image features to obtain a cascade feature vector;

[0031] S53. Use dimensionality reduction, clustering, classification and neural network model training to extract key features, and merge multiple key feature vectors into a multidimensional array.

[0032] Furthermore, in some embodiments, the step of performing association analysis between the data model and the structured data in step S6 includes:

[0033] S61, performing pre-processing operations such as cleaning and standardization on structured data from other sources;

[0034] Spatiotemporal fusion: adding the time and geographic location attributes of image objects to the data model;

[0035] Data fusion: Build the relationship between each data attribute and the key feature attributes of the image, and associate them to form a wide data table.

[0036] Furthermore, in some embodiments, the step of performing multidimensional data visualization analysis on the data wide table, drawing visualization area graphics and designing interactions in step S7 includes:

[0037] S71. Use time series analysis, spatial distribution analysis, correlation analysis, causal analysis and other methods to mine data value information.

[0038] S72. Select a corresponding visualization graphic based on the expression target of the information feature and draw it.

[0039] S73. Design visual graphic interaction logic to ensure that the drill-down information of the component graphic is associated with the image processed in step S4 and has the ability to be accessed at any time.

[0040] In order to better realize the present invention, further, the present invention also provides an intelligent data display system that integrates image processing and multi-dimensional visualization. The system includes a data source management module, an image preprocessing module, an image analysis and processing module, a multi-dimensional visualization analysis module, and a front-end display module. The supporting modules include a machine learning module, a data fusion module, a user interaction module, and a background management module.

[0041] The data source management module is used to access structured and unstructured data from different sources into the smart data display platform and perform basic preprocessing on the imported structured data;

[0042] Image preprocessing module, which cleans and enhances the original image and verifies the quality of the processed image;

[0043] The image analysis and processing module identifies the qualified images and extracts their features to convert them into data models;

[0044] The multi-dimensional visualization analysis module associates the feature data model with the structured data, selects appropriate visualization graphics to display the value of information, and designs interactions to facilitate users to explore data in multiple dimensions and levels.

[0045] The front-end display module provides a front-end user interface and supports multiple interaction methods;

[0046] To support the above modules, the system also includes:

[0047] Machine learning module, model training, model validation and model deployment;

[0048] Data fusion module, used for data association and spatiotemporal association;

[0049] User interaction module, used to design and implement front-end interactive operations such as drill-down, roll-up, query and feedback;

[0050] Backend management module, used for data management, user management, log auditing and security compliance management.

[0051] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0052] 1. The present invention realizes the effective processing and display of a large amount of image data by adopting an intelligent processing and analysis process of image data, improves the processing standardization and analysis quality of image data, and enhances the usability and interpretability of image data.

[0053] 2. The present invention uses multi-dimensional visual analysis to correlate key information in unstructured data with structured data, thereby deepening the utilization and value mining of image data.

[0054] 3. The intelligent visualization display of the present invention realizes the transformation from low-value-density graphic display to high-value-density graphic display, and realizes multi-level and multi-mode exploration of data through flexible interactive design. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings in the embodiment:

[0056] Figure 1 The figure is a flow chart of the intelligent data display method integrating image processing and multi-dimensional visualization according to the present invention.

[0057] Figure 2 This is a framework diagram of the intelligent data display system that integrates image processing and multi-dimensional visualization according to the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Therefore, it should not be regarded as a limitation on the scope of protection. For ordinary technicians in this field, all other embodiments obtained without creative work are within the scope of protection of the present invention.

[0059] Example 1:

[0060] Figure 1 This is a flow chart of a method provided by an embodiment of the present invention. The embodiment of the present invention provides an intelligent data display method integrating image processing and multi-dimensional visualization, comprising the following steps:

[0061] Step 1: Obtain a large amount of image information from drones, cameras, etc., and obtain data such as environment and equipment temperature, humidity, noise level, vibration, etc. from equipment sensors.

[0062] Step 2: Clean multiple original images. Specifically:

[0063] (1) Read the original image pixel matrix and input it into the pixel segmentation model to obtain multiple plane pixel regions after secondary optimization;

[0064] (2) construct an automatic encoder and use adaptive filtering, taking mean filtering, median filtering and bilateral filtering as alternatives in the spatial domain, and adjusting the appropriate denoising method and denoising parameters according to the image characteristics of each pixel area;

[0065] (3) Using the smooth window search method, unnecessary edge areas of the cleaned image are cropped.

[0066] Step 3: Image enhancement, specifically:

[0067] (1) Apply the gamma function s = cr γ Simulate the human eye's perception of brightness and eliminate image brightness inconsistencies caused by lighting differences;

[0068] (2) Adjust the pixel value distribution of the image through histogram equalization to enhance the contrast;

[0069] (3) Sharpening, using a high-pass filter to enhance image details and edges;

[0070] (4) Taking 1920×1080px as the standard, images below the resolution standard are reconstructed using a generative adversarial network for super-resolution, and images above the resolution standard are compressed to the standard level.

[0071] First, for the collected high-resolution images, the corresponding low-resolution images are generated to form the training set and the validation set;

[0072] Then design the architecture of the generator, which includes input layer, convolution layer, upsampling layer and output layer. The convolution layer can contain filters of different sizes, such as 3x3, 5x5. The upsampling layer uses a deconvolution layer or the nearest neighbor interpolation method to enlarge the image size. The loss function is the weighted sum of content loss and adversarial loss, where the content loss is measured by mean square error (MSE);

[0073] Then design the architecture of the discriminator, which includes input layer, convolution layer, fully connected layer and output layer. Use multiple convolution layers to extract image features. Fully connected layers are used to reduce dimensions and further process features. The output layer outputs the probability that the image is real. The loss function is measured using binary cross entropy loss.

[0074] Use Xavier or He initialization methods to initialize the weights in the neural network. During the training process, first fix the discriminator and only update the weights of the generator, so that the generator can generate realistic images as much as possible. Then fix the generator and update the weights of the discriminator to improve its ability to distinguish between true and false. Through continuous adversarial training between the two, the quality of the images obtained by the generator is continuously improved. As the training progresses, regularly check the changes in the loss of the generator and the discriminator, and gradually reduce the learning rate to avoid overfitting.

[0075] The visual quality of the generated images is checked using metrics such as PSNR and SSIM. When the model reaches a satisfactory performance standard, it is applied to the images processed in step S3.

[0076] Step 4: Evaluate the image quality and perform information entropy verification, average gradient verification, and full color verification.

[0077] If the image quality is not up to standard, the image is taken as an image to be processed and the process returns to the step of re-preprocessing the image until the image quality is up to standard.

[0078] Information entropy verification:

[0079] According to the information entropy calculation formula Calculate the information entropy of the image, where E represents the information entropy of the image, M×N represents the size of the processing window when processing the image, i, j represents the position of the processing window when processing the image, and P i is the probability of occurrence of each gray level / pixel value, and L is the total number of gray levels.

[0080] Average gradient check:

[0081] Calculate the formula based on the average gradient Calculate the average gradient of the image, where AG represents the average gradient of the image, M×N represents the size of the processing window when processing the image, i, j represents the location of the processing window when processing the image, and F x Indicates the difference value in the x direction, F y Indicates the difference value in the y direction.

[0082] Full color calibration:

[0083] The superpixel network uses the FCN architecture to perform superpixel segmentation on the image and calculates the standard deviation of each superpixel to estimate the contrast map:

[0084]

[0085] Among them, S 1,i represents the i-th superpixel of image I1, |S 1,i |Represents S 1,i The number of pixels, μ 1,i It is Super Pixel S 1,i The average value of the pixels in .

[0086] Step 5: Perform pattern recognition on the qualified images, extract the key features of the images and quantify them to obtain a data model.

[0087] (1) Gradient path and intensity path are used to extract image features.

[0088] Use color histogram to analyze the distribution of colors in the image;

[0089] Use the histogram of directional gradients to calculate and count the gradient changes of pixels in each area of ​​the image to form HOG features;

[0090] Use gray-level co-occurrence matrix algorithm or local binary pattern algorithm to describe the texture of the image;

[0091] Use edge detection operators such as Sobel and Canny to identify locations where brightness changes significantly, and detect the shape and outline of objects or the boundaries between different areas;

[0092] Use Harris, Shi-Tomasi and other methods to identify corners or key points in the image;

[0093] (2) The obtained image features are fused through a cascade method, and then through a convolution with a step size of 1, a convolution kernel size of 1×1, and a Tanh activation function The information is further merged to obtain the concatenated feature vector.

[0094] There are many ways to extract key features: use K-means, DBSCAN and other algorithms to cluster pixels or features in the image; use PCA dimensionality reduction algorithm to extract the main features of the image; use linear discriminant analysis to find the best feature classification boundary; use support vector machines, decision trees, neural networks and other deep learning models to identify complex pattern tasks. For example, use pre-trained deep neural network CNN to automatically learn and extract multi-level features of images, identify key components such as transformers, poles, and wires in power facilities, and automatically detect potential problems such as cracks, corrosion, and wear on power facilities.

[0095] Step 6: Perform correlation analysis between the data model and structured data from other sources to form a wide data table.

[0096] (1) Preprocess the structured data from other sources by cleaning and standardizing them, including filling missing values, removing duplicate values, processing outliers, character extraction, and other basic operations.

[0097] (2) Add the time and location attributes of the image object to the data model to analyze the changing trends of the data over time and location.

[0098] (3) Map the feature data model and sensor structured data into the same reference frame, and use a weighted fusion algorithm to combine the information to form a more complete description of the facility status. Construct the relationship between each data attribute and the key feature attributes of the image, and associate them to form a data wide table;

[0099] Step 7: Perform multidimensional data visualization analysis on the data wide table, draw visualization area graphics and design interactions;

[0100] (1) Use time series analysis, spatial distribution analysis, correlation analysis, causal analysis and other methods to mine data value information.

[0101] (2) Select the corresponding visualization graphics to be drawn based on the expression target of the information feature.

[0102] There are many methods for visualizing multidimensional data: scatter / bubble charts, which display two-dimensional or three-dimensional data; parallel coordinate charts and spatial coordinate charts, where each directional axis represents a dimension and data points are displayed across dimensions in the form of line segments or surfaces; heat maps, which display the relationship or similarity matrix between features; radar charts, which compare samples or categories under multidimensional attributes; and maps, which display the spatial distribution and changes of attribute data.

[0103] (3) Design the interactive logic of the visualization graphics to ensure that the drill-down information of the component graphics is associated with the image processed in step S4 and can be accessed at any time.

[0104] Step 8: Display data in an interactive user interface. Specifically, you can use interactive visualization tools, such as Tableau, which supports drag-and-drop interaction, by dragging and dropping large amounts of data onto a digital "canvas"; provide a variety of chart options, quickly create and modify charts through drag-and-drop operations, and configure component interactions through parameter passing and other methods.

[0105] The present invention also relates to an intelligent data display system integrating image processing and multi-dimensional visualization, which is used to more conveniently implement the present invention. The system includes a data source management module, an image preprocessing module, an image analysis and processing module, a multi-dimensional visualization analysis module, and a front-end display module. In addition, the support module includes a machine learning module, a data fusion module, a user interaction module, and a background management module.

[0106] The above is only an embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment according to the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. An intelligent data display method integrating image processing and multi-dimensional visualization, characterized in that: include: S1. Obtain multiple raw images and structured data to be processed from different sources; S2, cleaning multiple original images; S3, performing enhancement processing on the cleaned image to improve the image quality; S4. Verify image quality to ensure that the clarity, saturation and information content of the processed image meet the quality requirements; S5. Perform pattern recognition on the images that have passed the quality inspection, extract key features of the images and quantify them to obtain a data model; S6. Perform correlation analysis on the data model and structured data to form a wide data table; S7. Perform multidimensional data visualization analysis on wide data tables, draw visualization area graphics and design interactions; S8. Display data in an interactive user interface.

2. The intelligent data display method integrating image processing and multi-dimensional visualization according to claim 1, characterized in that: The S3 performs enhancement processing on the cleaned image to improve the image quality, including: Increase or decrease the overall brightness of the image to eliminate image brightness inconsistencies caused by lighting differences; The pixel value distribution of the image is adjusted by equalization to enhance the contrast; Enhance the details and edges of the image and perform sharpening; The image resolution is adjusted to the standard value. For low-resolution images, a generative adversarial network is used to reconstruct the image in super-resolution.

3. The intelligent data display method integrating image processing and multi-dimensional visualization according to claim 1, characterized in that: The S5 performs pattern recognition on the qualified images, extracts key features of the images and quantifies them to obtain a data model, including: Extract color, edge, and corner features from the preprocessed image; Cascading the obtained image features to obtain a cascade feature vector; Dimensionality reduction, clustering, classification and neural network model training are used to extract key features, and multiple key feature vectors are merged into a multidimensional array.

4. The intelligent data display method integrating image processing and multi-dimensional visualization according to claim 1, characterized in that: The step of performing association analysis between the data model and the structured data in S6 includes: Perform pre-processing operations such as cleaning and standardization on structured data from other sources; Spatiotemporal fusion: adding the time and geographic location attributes of image objects to the data model; Data fusion: Build the relationship between each data attribute and the key feature attributes of the image, and associate them to form a wide data table.

5. The intelligent data display method integrating image processing and multi-dimensional visualization according to claim 1, characterized in that: The steps of performing multidimensional data visualization analysis on the data wide table, drawing visualization area graphics and designing interactions include: Use time series analysis, spatial distribution analysis, correlation analysis, causal analysis and other methods to mine data value information; Select corresponding visualization graphics for drawing according to the expression target of information features; Design the interactive logic of the visualization graphics to ensure that the drill-down information of the component graphics is associated with the image processed in step S4 and can be accessed at any time.

6. An intelligent data display system integrating image processing and multi-dimensional visualization, characterized in that: include: A data source management module is used to obtain multiple raw images and structured data to be processed from different sources; An image preprocessing module is used to clean a plurality of original images to be processed, enhance the cleaned images, and verify the image quality; Image analysis and processing module, used to perform pattern recognition on images that have passed quality inspection, extract key features of the images and quantify them to obtain data models; Multi-dimensional visualization analysis module, used to associate data models with structured data to form a wide data table; Perform multi-dimensional data visualization analysis on wide data tables, draw visualization area graphics and design interactions; The front-end display module is used to display data in an interactive user interface.

7. The intelligent data display system integrating image processing and multi-dimensional visualization according to claim 6, characterized in that: Also includes: Machine learning module, model training, model validation and model deployment; Data fusion module, used for data association and spatiotemporal association; User interaction module, used to design and implement front-end interactive operations such as drill-down, roll-up, query and feedback; Backend management module, used for data management, user management, log auditing, security compliance management, etc.

Images