Data processing system and method based on GPU acceleration metering calculation

Abstract

The invention discloses a data processing system and method based on GPU acceleration metering calculation, and the system comprises an image data collection module, a data collection end association module, a data storage module, a data analysis module, a GPU processing module, a CPU processing module, and a data deduplication module. The same repeated image data existing in the same batch of image data is extracted before the image data is calculated, repeated calculation is not needed, and the time for calculating the image data by the GPU is saved; through the arrangement of the data acceleration calculation module, isolation processing during calculation of the image data is realized, shared data calculation processing can be performed on the image data except for calculation and isolation through other idle GPU calculation components in a sharing manner, and the calculation efficiency of the image data is automatically allocated; and data loss cannot be caused by sharing.

Description

Technical Field

[0001] The present invention relates to the field of GPU data processing technology, and in particular to a data processing system and method based on GPU accelerated metrology calculation. Background Art

[0002] A data processing system is a software or hardware system used to process and manage data. It provides a standardized and structured way to manage and use data to help users better understand and use data. A data processing system is a system that uses computers to process information. Through this system, data information is processed and sorted, and various analytical indicators are calculated and converted into information forms that are easy for people to accept. The processed information can also be stored. GPU: The core component of the system, specially designed for graphics processing and large-scale parallel computing. The GPU has thousands of small processors (stream processors) that can process large amounts of data and tasks at the same time. It is particularly suitable for workloads that require high parallelism, such as deep learning, scientific computing, and graphics rendering. CPU: Although the GPU is the main computing unit of the system, the CPU still plays an important role. The CPU is responsible for performing various control tasks of the system, such as operating system operation, task scheduling, and communication. At the same time, the CPU and GPU usually work together to give full play to their respective advantages. Memory and video memory: Memory is used to store data and programs processed by the CPU, while video memory is specifically used to store data and programs processed by the GPU. High-speed video memory can transfer data and calculate faster, thereby improving performance. The CPU is a serial architecture that is conducive to the X86 instruction set. It is more suitable for completing a task as quickly as possible from the design perspective. However, the CPU is less efficient when processing parallel computing because it has a small number of cores and is not suitable for processing tasks with high computing density, multiple concurrent threads, and frequent memory access due to its architectural characteristics. The task of the GPU is to synthesize and display millions of pixels on the screen, which requires processing millions of tasks at the same time. Therefore, it is designed to process many tasks in parallel. From the perspective of hardware design, the GPU is composed of thousands of smaller and more efficient cores. These cores are designed for processing multiple tasks at the same time and have a larger bandwidth, which enables it to play a greater role in multimedia processing. However, the existing data processing based on GPU accelerated quantitative computing still has insufficient efficiency in the computational processing of image data. Duplicate image data and blurred image data are prone to appear in the image data. The existing GPU does not have the ability to pre-process this type of image data when calculating image data, which easily causes a waste of GPU computing resources and affects the efficiency of calculating image data results. Summary of the invention

[0003] The purpose of the present invention is to provide a data processing system and method based on GPU accelerated metrological calculation, so as to solve the problem that the calculation processing of image data in the existing data processing based on GPU accelerated metrological calculation mentioned in the above background technology is still not efficient enough, repeated image data and blurred image data are prone to appear in the image data, and the existing GPU does not have the ability to pre-process this type of image data when calculating image data, which easily causes a waste of GPU computing resources and affects the efficiency of calculating the image data results.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical scheme: a data processing system based on GPU accelerated measurement calculation, comprising an image data acquisition module, a data acquisition terminal association module, a data storage module, a data analysis module, a GPU processing module, a CPU processing module, a data deduplication module, a processing-free data encryption module, a data accelerated calculation module, a data decryption module, a data preprocessing module, a GPU accelerated calculation module, a result analysis module, a data processing visualization module, a performance acceleration adjustment module, and a manual data change recording module, wherein the output end of the image data acquisition module is communicatively connected to the input end of the data acquisition terminal association module, the data acquisition terminal association module is bidirectionally connected to the data storage module, the output end of the data storage module is communicatively connected to the input end of the data analysis module, the data analysis module is bidirectionally connected to the GPU processing module, and the GPU processing module is bidirectionally connected to the CPU processing module. The processing module is bidirectionally connected, the output end of the CPU processing module is communicatively connected to the input end of the data deduplication module, the output end of the data deduplication module is communicatively connected to the input end of the processing-free data encryption module, the output end of the processing-free data encryption module is communicatively connected to the input end of the data acceleration calculation module, the output end of the data acceleration calculation module is communicatively connected to the input end of the data decryption module, the output end of the data decryption module is communicatively connected to the input end of the data preprocessing module, the data preprocessing module is bidirectionally connected to the GPU acceleration calculation module, the output end of the GPU acceleration calculation module is communicatively connected to the input end of the result analysis module, the output end of the result analysis module is communicatively connected to the input end of the data processing visualization module, the data processing visualization module is bidirectionally connected to the performance acceleration adjustment module, and the data processing visualization module is bidirectionally connected to the manual data change record module;

[0005] The image data acquisition module is used to acquire and process the image data that needs to be processed;

[0006] The data acquisition end association module is used to perform association processing on each external authenticated device of input image data;

[0007] The data storage module is used to temporarily store and manage data and to queue the collected data;

[0008] The data analysis module is used to divide the image data in the data storage module into original image data and allocate tasks;

[0009] The GPU processing module is used to perform parallel computing processing on the graphic data through the small processor of the GPU;

[0010] The CPU processing module is used to execute various control tasks of data processing;

[0011] The data deduplication module is used to remove the same duplicate data in the same batch of image data to complete the deduplication work of data processing;

[0012] The processing-free data encryption module is used to set the encryption code and perform code encryption protection processing on repeated processing-free image data;

[0013] The data acceleration calculation module is used to perform isolated calculation on the image data, perform shared processing on the non-isolated image data, and complete synchronous shared calculation processing after sharing;

[0014] The data decryption module is used to decrypt the code-encrypted free-processing image data;

[0015] The data preprocessing module is used to integrate the image data and eliminate data redundancy;

[0016] The GPU accelerated computing module is used to accelerate the computing of image data;

[0017] The result analysis module is used to establish a verification mechanism, use checksums and hash values ​​to verify the accuracy of data results, ensure that no errors or damage occur during data transmission and processing, compare the calculation results with the expected results, and use different algorithms for verification and verification processing;

[0018] The data processing visualization module is used to intuitively present the processed data results in the form of charts, images, and animations, and personnel can adjust the algorithm and parameter settings through charts, images, and animations;

[0019] The performance acceleration adjustment module is used to manage the memory in the GPU program and adjust the computing core of the GPU;

[0020] The manual data change recording module performs recording processing when manual changes are made to the GPU computing data management.

[0021] Preferably, the data analysis module comprises a data task decomposition unit and a dynamic task allocation unit, and the output end of the data task decomposition unit is communicatively connected with the input end of the dynamic task allocation unit;

[0022] The data task decomposition unit is used to divide the original graph data and distribute the divided data to each GPU device;

[0023] The dynamic task allocation unit is used to estimate the size of the task volume, divide the tasks into different groups, and allocate computing resources according to the task volume to allocate the task volume.

[0024] Preferably, the data deduplication module comprises a duplicate data recording unit and a duplicate data free processing unit, and the output end of the duplicate data recording unit is communicatively connected to the input end of the duplicate data free processing unit;

[0025] The repeated data recording unit is used to record and process the data with repeated situations in the current data;

[0026] The duplicate data free processing unit is used to perform calculation-free processing operations on the recorded duplicate data.

[0027] Preferably, the data acceleration computing module comprises a data processing isolation unit and a data processing sharing unit, and the output end of the data processing isolation unit is communicatively connected to the input end of the data processing sharing unit;

[0028] The data processing isolation unit is used to isolate and calculate the image data;

[0029] The data processing sharing unit is used to share image data that is not isolated and calculated, and to process the shared image data through an idle GPU.

[0030] Preferably, the data decryption module comprises a data encryption and decryption unit and a data decryption unit, and the data encryption and decryption unit is bidirectionally connected with the data decryption unit;

[0031] The data encryption and decryption unit is used to synchronously store the data encryption code and obtain the code;

[0032] The data decryption unit is used to decrypt the encrypted processing-free image data through the acquired code.

[0033] Preferably, the data preprocessing module comprises an image data cleaning unit, an image data integration unit and an image data redundancy elimination unit, wherein the image data cleaning unit is bidirectionally connected to the image data integration unit, and the image data integration unit is bidirectionally connected to the image data redundancy elimination unit;

[0034] The image data cleaning unit is used to remove noise, abnormal values, duplicate values ​​and missing values ​​in the image data through cleaning, conversion and integration operations;

[0035] The image data integration unit is used to integrate the image data into a consistent format and specification;

[0036] The image data redundancy elimination unit is used to eliminate redundancy in the image data.

[0037] Preferably, the GPU accelerated computing module includes a deep learning unit, a data computing unit, a matrix algorithm unit, and a convolution operation unit;

[0038] The deep learning unit is used to shorten the training and inference time of the deep neural network;

[0039] The data computing unit is used to perform computing processing on the data through the GPU;

[0040] The matrix arithmetic unit is used to perform matrix operations to arrange the image data into a set of complex or real numbers in a rectangular array;

[0041] The convolution operation unit is used to perform convolution operation processing on image data.

[0042] Preferably, the manual data change record module includes a processing log unit, a manual data processing unit, an information verification unit, and a data backup and recovery unit, wherein the processing log unit is bidirectionally connected to the manual data processing unit, and the manual data processing unit is bidirectionally connected to the information verification unit;

[0043] The processing log unit is used to record the process of processing each image data;

[0044] The manual data processing unit is used to manually process the image data;

[0045] The information verification unit is used to perform identity information verification before manually processing image data.

[0046] Preferably, the manual data change recording module is bidirectionally connected to a data backup and recovery module, and the data backup and recovery module is used to record and back up the original data before image data processing, and perform backup and recovery processing when there is malicious image data.

[0047] The data processing method based on GPU accelerated metrology calculation includes the following steps:

[0048] S1, collecting image data through various collection ports;

[0049] S2, associating the CPU processor and the GPU processor;

[0050] S3, performing parallel computing processing on the collected image data through a small processor of a GPU processor;

[0051] S4, executing and controlling the control task through the CPU processor associated with the GPU processor;

[0052] S5, classifying the same image data and performing deduplication processing;

[0053] S6, performing image cleaning, conversion and integration operations on image data that do not have the same situation;

[0054] S7, performing deep learning, data calculation, matrix operation, and convolution operation on the image data;

[0055] S8. Visualize the calculated data results and change the running program according to the results.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows: by setting up a data deduplication module, it is possible to extract the same repeated image data existing in the same batch of image data before calculating the image data, without repeated calculation, thus saving the time of GPU calculating the image data;

[0058] By setting up a data acceleration computing module, image data can be isolated during computing. Image data outside of computing isolation can be shared through other idle GPU computing components for shared data computing processing, and the efficiency of image data computing can be automatically allocated. Sharing will not cause data loss.

[0059] By setting up a manual data change record module, it is possible to record the process of manual intervention when the GPU calculates image data. When the image data is maliciously tampered with, the original image data backed up in real time can be directly replaced and restored. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the drawings required for the implementation methods or the prior art descriptions are briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0061] Figure 1 This is a block diagram of the overall system of the present invention. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0063] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Embodiment 1:

[0065] See also Figure 1 As shown, the data processing system based on GPU accelerated metering calculation of the embodiment of the present invention includes an image data acquisition module, a data acquisition terminal association module, a data storage module, a data analysis module, a GPU processing module, a CPU processing module, a data deduplication module, a processing-free data encryption module, a data accelerated calculation module, a data decryption module, a data preprocessing module, a GPU accelerated calculation module, a result analysis module, a data processing visualization module, a performance acceleration adjustment module, and a manual data change recording module. The output end of the image data acquisition module is communicatively connected to the input end of the data acquisition terminal association module, the data acquisition terminal association module is bidirectionally connected to the data storage module, the output end of the data storage module is communicatively connected to the input end of the data analysis module, the data analysis module is bidirectionally connected to the GPU processing module, and the GPU processing module is bidirectionally connected to the CPU processing module. The processing module is bidirectionally connected, the output end of the CPU processing module is communicatively connected to the input end of the data deduplication module, the output end of the data deduplication module is communicatively connected to the input end of the processing-free data encryption module, the output end of the processing-free data encryption module is communicatively connected to the input end of the data acceleration calculation module, the output end of the data acceleration calculation module is communicatively connected to the input end of the data decryption module, the output end of the data decryption module is communicatively connected to the input end of the data preprocessing module, the data preprocessing module is bidirectionally connected to the GPU acceleration calculation module, the output end of the GPU acceleration calculation module is communicatively connected to the input end of the result analysis module, the output end of the result analysis module is communicatively connected to the input end of the data processing visualization module, the data processing visualization module is bidirectionally connected to the performance acceleration adjustment module, and the data processing visualization module is bidirectionally connected to the manual data change record module;

[0066] The image data acquisition module is used to acquire and process the image data that needs to be processed;

[0067] The data acquisition end association module is used to associate the authenticated devices of each external input image data;

[0068] The data storage module is used to temporarily store and manage data and queue the collected data;

[0069] The data analysis module is used to divide the original image data in the data storage module and allocate tasks;

[0070] The GPU processing module is used to perform parallel computing on the graphics data through the small processor of the GPU;

[0071] The CPU processing module is used to execute various control tasks for data processing;

[0072] The data deduplication module is used to remove the same duplicate data in the same batch of image data to complete the deduplication work of data processing;

[0073] By setting up a data deduplication module, it is possible to extract the same duplicate image data in the same batch of image data before calculating the image data, eliminating the need for repeated calculations and saving the time of GPU calculating image data;

[0074] The processing-free data encryption module is used to set the encryption code and perform code encryption protection processing on repeated processing-free image data;

[0075] The data acceleration computing module is used to perform isolated computing on the image data, and to perform shared processing on the non-isolated image data, and to complete the synchronous shared computing processing after sharing;

[0076] By setting up a data acceleration computing module, image data can be isolated during computing. Image data outside of computing isolation can be shared through other idle GPU computing components for shared data computing processing, and the efficiency of image data computing can be automatically allocated. Sharing will not cause data loss.

[0077] The data decryption module is used to decrypt the code-encrypted free-processing image data;

[0078] The data preprocessing module is used to integrate the image data and eliminate data redundancy;

[0079] The GPU accelerated computing module is used to accelerate the computing of image data;

[0080] The result analysis module is used to establish a verification mechanism, use checksums and hash values ​​to verify the accuracy of data results, ensure that there are no errors or damage to the data during transmission and processing, compare the calculation results with the expected results, and use different algorithms for verification and verification;

[0081] The data processing visualization module is used to intuitively present the processed data results in the form of charts, images, and animations. Personnel can adjust the algorithm and parameter settings through charts, images, and animations.

[0082] The performance acceleration adjustment module is used to manage the memory in the GPU program and adjust the GPU's computing core;

[0083] The manual data change record module records and processes manual changes to GPU computing data management;

[0084] By setting up a manual data change record module, it is possible to record the process of manual intervention when the GPU calculates image data. When the image data is maliciously tampered with, the original image data backed up in real time can be directly replaced and restored.

[0085] Embodiment 2:

[0086] The data analysis module includes a data task decomposition unit and a dynamic task allocation unit, and the output end of the data task decomposition unit is communicatively connected with the input end of the dynamic task allocation unit;

[0087] The data task decomposition unit is used to divide the original graph data and distribute the divided data to each GPU device;

[0088] The dynamic task allocation unit is used to estimate the size of the task volume, divide the tasks into different groups, and allocate computing resources according to the task volume to distribute the task volume.

[0089] The data deduplication module includes a duplicate data recording unit and a duplicate data free processing unit, and the output end of the duplicate data recording unit is communicatively connected with the input end of the duplicate data free processing unit;

[0090] The duplicate data recording unit is used to record and process the data with duplicate conditions in the current data;

[0091] The duplicate data free processing unit is used to perform calculation-free processing operations on the recorded duplicate data;

[0092] By setting up a data deduplication module, it is possible to extract the same duplicate image data in the same batch of image data before calculating the image data, eliminating the need for repeated calculations and saving the time of GPU calculating image data.

[0093] The data acceleration computing module includes a data processing isolation unit and a data processing sharing unit, and the output end of the data processing isolation unit is communicatively connected to the input end of the data processing sharing unit;

[0094] The data processing isolation unit is used to isolate and calculate the image data;

[0095] The data processing sharing unit is used to share the image data that is not isolated and calculated, and to process the shared image data through the idle GPU;

[0096] By setting up a data acceleration computing module, isolated processing is achieved during image data calculation. Image data outside the calculation isolation can be shared through other idle GPU computing components for shared data calculation processing, and the efficiency of image data calculation is automatically allocated. Sharing will not cause data loss.

[0097] Wherein, the data decryption module includes a data encryption and decryption unit and a data decryption unit, and the data encryption and decryption unit is bidirectionally connected with the data decryption unit;

[0098] The data encryption and decryption unit is used to synchronously store the data encryption code and obtain the code;

[0099] The data decryption unit is used to decrypt the encrypted free-processing image data through the acquired code.

[0100] The data preprocessing module includes an image data cleaning unit, an image data integration unit and an image data redundancy elimination unit. The image data cleaning unit is bidirectionally connected to the image data integration unit, and the image data integration unit is bidirectionally connected to the image data redundancy elimination unit.

[0101] The image data cleaning unit is used to remove noise, abnormal values, duplicate values ​​and missing values ​​in the image data through cleaning, conversion and integration operations;

[0102] The image data integration unit is used to integrate the image data into a consistent format and specification;

[0103] The image data redundancy elimination unit is used to eliminate redundancy in the image data.

[0104] Among them, the GPU accelerated computing module includes a deep learning unit, a data computing unit, a matrix algorithm unit, and a convolution operation unit;

[0105] The deep learning unit is used to shorten the training and inference time of deep neural networks;

[0106] The data computing unit is used to perform computing processing on the data through the GPU;

[0107] The matrix arithmetic unit is used to perform matrix operations, which arranges the image data into a set of complex or real numbers in a rectangular array;

[0108] The convolution operation unit is used to perform convolution operation processing on image data.

[0109] Among them, the manual data change record module includes a processing log unit, a manual data processing unit, an information verification unit, and a data backup and recovery unit. The processing log unit is bidirectionally connected to the manual data processing unit, and the manual data processing unit is bidirectionally connected to the information verification unit.

[0110] The processing log unit is used to record the process of processing each image data;

[0111] The manual data processing unit is used to manually process the image data;

[0112] The information verification unit is used to perform identity information verification before manually processing image data;

[0113] When the GPU calculates image data, the process records the cases where there is human intervention. When the image data is maliciously tampered with, the original image data backed up in real time can be directly replaced and restored.

[0114] The manual data change record module is bidirectionally connected to the data backup and recovery module, and the data backup and recovery module is used to record and back up the original data before the image data is processed, and perform backup and recovery processing when there is malicious image data.

[0115] Embodiment 3:

[0116] The data processing method based on GPU accelerated metrology calculation includes the following steps:

[0117] S1, collecting image data through various collection ports;

[0118] S2, associating the CPU processor and the GPU processor;

[0119] S3, performing parallel computing processing on the collected image data through a small processor of a GPU processor;

[0120] S4, executing and controlling the control task through the CPU processor associated with the GPU processor;

[0121] S5, classifying the same image data and performing deduplication processing;

[0122] S6, performing image cleaning, conversion and integration operations on image data that do not have the same situation;

[0123] S7, performing deep learning, data calculation, matrix operation, and convolution operation on the image data;

[0124] S8. Visualize the calculated data results and change the running program according to the results.

[0125] Embodiment 4:

[0126] By associating the authenticated devices of each external input image data, the image data to be processed is collected through the authenticated devices, the original image data is divided, and the divided data is distributed to each GPU device, and then the size of the task is estimated, the tasks are divided into different groups, and computing resources are allocated according to the task amount. The task amount is allocated, parallel computing is performed through the GPU, and each task is controlled and executed through the CPU. When there is repeated image data in the image data, the image data is free of computing and isolated when the image data is calculated. The image data outside the computational isolation can be shared through other idle GPU computing components for shared data computing processing. The efficiency of image data computing is automatically allocated, and sharing will not cause data loss. The image data is cleaned, converted and integrated to complete the removal of noise, outliers, duplicate values ​​and missing values ​​in the image data, integrate the image data into a consistent format and specification, eliminate redundancy in the image data, and complete the calculation of the image data through deep learning, data calculation, matrix algorithm, and convolution operation.

[0127] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0128] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope of the present disclosure is indicated by the following claims.

Claims

1. A data processing system based on GPU accelerated metrology calculation, characterized in that: The system comprises an image data acquisition module, a data acquisition terminal association module, a data storage module, a data analysis module, a GPU processing module, a CPU processing module, a data deduplication module, a processing-free data encryption module, a data acceleration calculation module, a data decryption module, a data preprocessing module, a GPU acceleration calculation module, a result analysis module, a data processing visualization module, a performance acceleration adjustment module, and a manual data change recording module. The output end of the image data acquisition module is communicatively connected to the input end of the data acquisition terminal association module, the data acquisition terminal association module is bidirectionally connected to the data storage module, the output end of the data storage module is communicatively connected to the input end of the data analysis module, the data analysis module is bidirectionally connected to the GPU processing module, the GPU processing module is bidirectionally connected to the CPU processing module, and the output end of the CPU processing module is bidirectionally connected. The end is communicatively connected to the input end of the data deduplication module, the output end of the data deduplication module is communicatively connected to the input end of the processing-free data encryption module, the output end of the processing-free data encryption module is communicatively connected to the input end of the data acceleration calculation module, the output end of the data acceleration calculation module is communicatively connected to the input end of the data decryption module, the output end of the data decryption module is communicatively connected to the input end of the data preprocessing module, the data preprocessing module is bidirectionally connected to the GPU acceleration calculation module, the output end of the GPU acceleration calculation module is communicatively connected to the input end of the result analysis module, the output end of the result analysis module is communicatively connected to the input end of the data processing visualization module, the data processing visualization module is bidirectionally connected to the performance acceleration adjustment module, and the data processing visualization module is bidirectionally connected to the manual data change record module; The image data acquisition module is used to acquire and process the image data that needs to be processed; The data acquisition end association module is used to perform association processing on each external authenticated device of input image data; The data storage module is used to temporarily store and manage data and to queue the collected data; The data analysis module is used to divide the image data in the data storage module into original image data and allocate tasks; The GPU processing module is used to perform parallel computing processing on the graphic data through the small processor of the GPU; The CPU processing module is used to execute various control tasks of data processing; The data deduplication module is used to remove the same duplicate data in the same batch of image data to complete the deduplication work of data processing; The processing-free data encryption module is used to set the encryption code and perform code encryption protection processing on repeated processing-free image data; The data acceleration calculation module is used to perform isolated calculation on the image data, perform shared processing on the non-isolated image data, and complete synchronous shared calculation processing after sharing; The data decryption module is used to decrypt the code-encrypted free-processing image data; The data preprocessing module is used to integrate the image data and eliminate data redundancy; The GPU accelerated computing module is used to accelerate the computing of image data; The result analysis module is used to establish a verification mechanism, use checksums and hash values ​​to verify the accuracy of data results, ensure that no errors or damage occur during data transmission and processing, compare the calculation results with the expected results, and use different algorithms for verification and verification processing; The data processing visualization module is used to intuitively present the processed data results in the form of charts, images, and animations, and personnel can adjust the algorithm and parameter settings through charts, images, and animations; The performance acceleration adjustment module is used to manage the memory in the GPU program and adjust the computing core of the GPU; The manual data change recording module performs recording processing when manual changes are made to the GPU computing data management.

2. The data processing system based on GPU accelerated metrology calculation according to claim 1, characterized in that: The data analysis module includes a data task decomposition unit and a dynamic task allocation unit, wherein the output end of the data task decomposition unit is communicatively connected to the input end of the dynamic task allocation unit; The data task decomposition unit is used to divide the original graph data and distribute the divided data to each GPU device; The dynamic task allocation unit is used to estimate the size of the task volume, divide the tasks into different groups, and allocate computing resources according to the task volume to allocate the task volume.

3. The data processing system based on GPU accelerated metrology calculation according to claim 2, characterized in that: The data deduplication module includes a duplicate data recording unit and a duplicate data processing-free unit, and the output end of the duplicate data recording unit is communicatively connected to the input end of the duplicate data processing-free unit; The repeated data recording unit is used to record and process the data with repeated situations in the current data; The duplicate data free processing unit is used to perform calculation-free processing operations on the recorded duplicate data.

4. The data processing system based on GPU accelerated metrology calculation according to claim 3, characterized in that: The data acceleration computing module includes a data processing isolation unit and a data processing sharing unit, and the output end of the data processing isolation unit is communicatively connected to the input end of the data processing sharing unit; The data processing isolation unit is used to isolate and calculate the image data; The data processing sharing unit is used to share image data that is not isolated and calculated, and to process the shared image data through an idle GPU.

5. The data processing system based on GPU accelerated metrology calculation according to claim 4, characterized in that: The data decryption module includes a data encryption and decryption unit and a data decryption unit, and the data encryption and decryption unit is bidirectionally connected with the data decryption unit; The data encryption and decryption unit is used to synchronously store the data encryption code and obtain the code; The data decryption unit is used to decrypt the encrypted processing-free image data through the acquired code.

6. The data processing system based on GPU accelerated metrology calculation according to claim 5, characterized in that: The data preprocessing module includes an image data cleaning unit, an image data integration unit and an image data redundancy elimination unit, wherein the image data cleaning unit is bidirectionally connected to the image data integration unit, and the image data integration unit is bidirectionally connected to the image data redundancy elimination unit; The image data cleaning unit is used to remove noise, abnormal values, duplicate values ​​and missing values ​​in the image data through cleaning, conversion and integration operations; The image data integration unit is used to integrate the image data into a consistent format and specification; The image data redundancy elimination unit is used to eliminate redundancy in the image data.

7. The data processing system based on GPU accelerated metrology calculation according to claim 6, characterized in that: The GPU accelerated computing module includes a deep learning unit, a data computing unit, a matrix algorithm unit, and a convolution operation unit; The deep learning unit is used to shorten the training and inference time of the deep neural network; The data computing unit is used to perform computing processing on the data through the GPU; The matrix arithmetic unit is used to perform matrix operations to arrange the image data into a set of complex or real numbers in a rectangular array; The convolution operation unit is used to perform convolution operation processing on image data.

8. The data processing system based on GPU accelerated metrology calculation according to claim 7, characterized in that: The manual data change record module includes a processing log unit, a manual data processing unit, an information verification unit, and a data backup and recovery unit. The processing log unit is bidirectionally connected to the manual data processing unit, and the manual data processing unit is bidirectionally connected to the information verification unit. The processing log unit is used to record the process of processing each image data; The manual data processing unit is used to manually process the image data; The information verification unit is used to perform identity information verification before manually processing image data.

9. The data processing system based on GPU accelerated metrology calculation according to claim 8, characterized in that: The manual data change recording module is bidirectionally connected to a data backup and recovery module, and the data backup and recovery module is used to record and back up the original data before the image data is processed, and perform backup and recovery processing when there is malicious image data.

10. A data processing method based on GPU accelerated metrology calculation, characterized in that: The following steps are involved: S1, collecting image data through various collection ports; S2, associating the CPU processor and the GPU processor; S3, performing parallel computing processing on the collected image data through a small processor of a GPU processor; S4, executing and controlling the control task through the CPU processor associated with the GPU processor; S5, classifying the same image data and performing deduplication processing; S6, performing image cleaning, conversion and integration operations on image data that do not have the same situation; S7, performing deep learning, data calculation, matrix operation, and convolution operation on the image data; S8. Visualize the calculated data results and change the running program according to the results.

Images