A Data Optimization Method and Apparatus Based on Multi-Source Heterogeneous Time-Series Auxiliary

Abstract

This application discloses a data optimization method and apparatus based on multi-source heterogeneous time-series assistance, belonging to the field of data optimization technology. The method includes: acquiring current and historical examination data of a target user; converting paper image data into electronic image data through image digitization; dividing all electronic image data into homogeneous data and multi-source heterogeneous data, and preprocessing both to obtain preprocessed historical examination data; establishing a three-dimensional model based on the target organ corresponding to the current examination data and performing mesh generation to obtain a three-dimensional model including multiple meshes; inputting the preprocessed historical examination data into the three-dimensional model, and assigning weights to each mesh in the three-dimensional model to obtain a target three-dimensional model; generating predicted examination data based on the target three-dimensional model, and comparing and optimizing the current examination data to obtain the target examination data. This method improves the accuracy and authenticity of the data.

Description

Technical Field

[0001] This application belongs to the field of data optimization technology, and in particular relates to a data optimization method and apparatus based on multi-source heterogeneous time-series assistance. Background Technology

[0002] Endoscopic ultrasound (EUS) is a medical device with continuously advancing technology and expanding functions, filling gaps in coverage for specific indications that cannot be addressed by conventional endoscopes, surface ultrasound, and CT scans. In recent years, with the continuous development of EUS technology, its application in the medical field has become increasingly widespread, serving as an important means of examining various conditions. However, current endoscopic ultrasound examinations still face many challenges, particularly in image quality and data processing.

[0003] Current endoscopic ultrasound technology is limited by endoscope size and ultrasound sensor performance, resulting in lower real-time image resolution and higher noise levels, which poses challenges for diagnosis and assessment. Because the size of the endoscope needs to be strictly controlled, and the performance of the ultrasound sensor is not yet ideal, the clarity and detail of endoscopic ultrasound images are often inferior to other imaging devices in practical applications, affecting the accuracy of physicians' lesion assessments. Furthermore, endoscopic ultrasound primarily enters the target area through natural body cavities for localized observation. This localized observation method makes it difficult to construct a real-time three-dimensional model of the organ in the same dimension, thus affecting the assessment of the overall organ condition. While endoscopic ultrasound can provide detailed observation of target organs, its limitation lies in providing only two-dimensional images, making it difficult to provide complete three-dimensional spatial information, which poses a challenge for the diagnosis and treatment planning of complex lesions.

[0004] Current technologies for endoscopic ultrasound examinations are susceptible to variations in results due to differences in equipment, operators, and environment, making efficient longitudinal comparisons impossible. This inconsistency not only increases the workload but also affects the continuity and accuracy of diagnosis. Existing examination data suffers from low accuracy and continuity. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a data optimization method based on multi-source heterogeneous time-series assistance, which improves the accuracy and authenticity of the data.

[0006] In a first aspect, this application provides a data optimization method based on multi-source heterogeneous time-series assistance, the method comprising:

[0007] Obtain the target user's current inspection data and historical inspection data. The historical inspection data includes electronic image data and paper image data. Convert the paper image data into electronic image data through image digitization.

[0008] All electronic image data are divided into homogeneous data and heterogeneous multi-source data. The homogeneous data and heterogeneous multi-source data are preprocessed separately to obtain preprocessed historical inspection data.

[0009] A three-dimensional model is built based on the target organ corresponding to the current examination data and meshed to obtain a three-dimensional model including multiple meshes;

[0010] The preprocessed historical inspection data is input into the 3D model, and weights are assigned to each grid in the 3D model to obtain the target 3D model.

[0011] Predictive inspection data is generated based on the target 3D model, and the current inspection data is compared and optimized to obtain the target inspection data.

[0012] According to one embodiment of this application, the step of converting paper image data into electronic image data through image digitization includes:

[0013] Determine whether electronic image data and paper image data in historical inspection data are duplicates. If electronic image data and paper image data are duplicates, delete the paper image data that is duplicated with the electronic image data. If electronic image data and paper image data are not duplicates, retain both electronic image data and paper image data.

[0014] All paper image data are scanned to obtain electronic image data in PNG format;

[0015] After cropping each PNG format electronic image data, the signal-to-noise ratio is calculated. It is then determined whether the signal-to-noise ratio is lower than a preset threshold. Electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered, while electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

[0016] If the signal-to-noise ratio of the filtered electronic image data is lower than the preset threshold, the electronic image data is discarded, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

[0017] According to one embodiment of this application, the step of dividing all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocessing the homogeneous data and heterogeneous multi-source data respectively to obtain preprocessed historical inspection data includes:

[0018] All electronic image data are compared with the parameters of the current examination data. Electronic image data with the same sensor type and resolution as the current examination data are selected as source data. The source data are numbered with 10 digits, where the first 8 digits are the image acquisition time and the last 2 digits are the block number corresponding to the three-dimensional organ represented by the image. The preprocessed source data is obtained.

[0019] Electronic image data other than data from the same source are treated as multi-source heterogeneous data, and it is determined whether the type and resolution of the multi-source heterogeneous data are the same as those of the currently inspected data.

[0020] If the multi-source heterogeneous data is of a different type than the currently inspected data, convert the multi-source heterogeneous data into data of the same type as the currently inspected data.

[0021] If the resolution of the multi-source heterogeneous data is different from that of the current inspection data, the multi-source heterogeneous data will be converted into data with the same resolution as the current inspection data.

[0022] The multi-source heterogeneous data are numbered 10 times, with the first 8 digits representing the image acquisition time and the last 2 digits representing the block number corresponding to the three-dimensional organ represented by the image, thus obtaining the preprocessed multi-source heterogeneous data.

[0023] According to one embodiment of this application, the step of establishing a three-dimensional model of the target organ based on the current examination data and performing mesh division to obtain a three-dimensional model including multiple meshes includes:

[0024] Construct a three-dimensional model of the target organ corresponding to the current examination data;

[0025] The three-dimensional model is meshed to obtain multiple meshes;

[0026] Each grid is numbered. Based on the relationship between the last two digits of the preprocessed historical inspection data block number and the grid number, the preprocessed homogeneous data and multi-source heterogeneous data are applied to the surface of the 3D model corresponding to the grid as surface textures, resulting in a 3D model including multiple grids.

[0027] According to one embodiment of this application, the step of inputting preprocessed historical inspection data into a 3D model and assigning weights to each mesh in the 3D model to obtain a target 3D model includes:

[0028] The preprocessed historical inspection data is input into the 3D model, and the historical inspection data of each grid is numbered.

[0029] Calculate the time interval between the historical inspection data and the current inspection data for each grid;

[0030] The historical inspection data number for each grid is updated based on the aforementioned time interval;

[0031] A weighting formula is constructed based on the updated historical inspection data numbering, and weights are assigned to each grid in the 3D model to obtain the target 3D model.

[0032] According to one embodiment of this application, the step of generating predictive inspection data based on the target 3D model and comparing and optimizing the current inspection data to obtain target inspection data includes:

[0033] A time-series 3D visualization model of multiple local parts is established based on the target 3D model;

[0034] Predictive inspection data is generated based on a time-series 3D visualization model of multiple local parts. The current inspection data is then compared and optimized to obtain the target inspection data.

[0035] According to one embodiment of this application, the step of generating predictive inspection data based on a temporal three-dimensional visualization model of multiple local parts, comparing and optimizing the current inspection data to obtain target inspection data includes:

[0036] Rotate the time-series 3D visualization models of multiple local parts to the same viewpoint and capture multiple images;

[0037] Weights are assigned to multiple images based on a weighting formula to generate predictive inspection data.

[0038] Calculate the mean square error and structural similarity index between the predicted inspection data and the current inspection data, compare and optimize the current inspection data to obtain the target inspection data.

[0039] Secondly, this application provides a data optimization device based on multi-source heterogeneous time-series assistance, the device comprising:

[0040] The acquisition module is used to acquire the target user's current inspection data and historical inspection data. The historical inspection data includes electronic image data and paper image data. The paper image data is converted into electronic image data through image digitization processing.

[0041] The first processing module is used to divide all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocess the homogeneous data and heterogeneous multi-source data respectively to obtain preprocessed historical inspection data.

[0042] The second processing module is used to build a three-dimensional model of the target organ corresponding to the current examination data and perform mesh division to obtain a three-dimensional model including multiple meshes.

[0043] The third processing module is used to input the preprocessed historical inspection data into the 3D model, assign weights to each grid in the 3D model, and obtain the target 3D model.

[0044] The optimization module is used to generate predictive inspection data based on the target 3D model, compare and optimize the current inspection data to obtain the target inspection data.

[0045] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data optimization method based on multi-source heterogeneous timing assistance as described in the first aspect above.

[0046] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data optimization method based on multi-source heterogeneous timing assistance as described in the first aspect above.

[0047] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the data optimization method based on multi-source heterogeneous timing assistance as described in the first aspect.

[0048] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the data optimization method based on multi-source heterogeneous timing assistance as described in the first aspect above.

[0049] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0050] The present invention provides a data optimization method based on multi-source heterogeneous time-series assistance, which has the following advantages over the prior art:

[0051] (1) This invention obtains the current and historical examination data of the target user and digitizes the paper image data into electronic image data, which can effectively integrate homogeneous data and heterogeneous multi-source data, thereby improving the quality and consistency of the data. A three-dimensional model is established based on the target organ corresponding to the current examination data, and the model is divided into grids and weighted. By inputting historical data into the three-dimensional model to generate predicted examination data and comparing and optimizing it with the current examination data, the accuracy and authenticity of the examination data are improved.

[0052] (2) This invention effectively improves the quality and consistency of data by classifying all electronic image data and processing homogeneous data and heterogeneous multi-source data separately. By screening out homogeneous data with the same sensor type and resolution as the current inspection data and numbering them, the accuracy and traceability of homogeneous data are improved. By converting heterogeneous multi-source data of different types or resolutions to maintain consistency with the current inspection data, multi-source data can be effectively integrated, improving the efficiency and reliability of data processing.

[0053] (3) By inputting preprocessed historical inspection data into a three-dimensional model and numbering and calculating time intervals for each grid, this invention can more accurately reflect the time difference between each grid and the current inspection data. Based on the updated historical inspection data number, a weight allocation formula is constructed and weights are allocated, realizing the auxiliary detection of historical data in local areas. By considering the time factor, the processing of historical data is more timely, and the impact of data in each grid on the current inspection data can be better evaluated. Attached Figure Description

[0054] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0055] Figure 1 This is a flowchart illustrating the data optimization method based on multi-source heterogeneous time-series assistance provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the structure of the data optimization device based on multi-source heterogeneous timing assistance provided in the embodiments of this application;

[0057] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0059] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0060] The following description, in conjunction with the accompanying drawings, details the data optimization method, device, electronic device, and readable storage medium based on multi-source heterogeneous timing assistance provided in this application, through specific embodiments and application scenarios.

[0061] Among them, the data optimization method based on multi-source heterogeneous timing assistance can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0062] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0063] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0064] The data optimization method based on multi-source heterogeneous timing assistance provided in this application embodiment can be executed by an electronic device or a functional module or functional entity in an electronic device that can implement the data optimization method based on multi-source heterogeneous timing assistance. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The following uses an electronic device as the execution subject to illustrate the data optimization method based on multi-source heterogeneous timing assistance provided in this application embodiment.

[0065] Figure 1 This is a flowchart illustrating the data optimization method based on multi-source heterogeneous time-series assistance provided in an embodiment of this application, as shown below. Figure 1 As shown, the data optimization method based on multi-source heterogeneous time-series assistance includes steps 110, 120, 130, 140 and 150.

[0066] Step 110: Obtain the target user's current inspection data and historical inspection data. The historical inspection data includes electronic image data and paper image data. Convert the paper image data into electronic image data through image digitization.

[0067] In some embodiments, the process of converting paper image data into electronic image data through image digitization includes:

[0068] Determine whether electronic image data and paper image data in historical inspection data are duplicates. If electronic image data and paper image data are duplicates, delete the paper image data that is duplicated with the electronic image data. If electronic image data and paper image data are not duplicates, retain both electronic image data and paper image data.

[0069] All paper image data are scanned to obtain electronic image data in PNG format;

[0070] After cropping each PNG format electronic image data, the signal-to-noise ratio is calculated. It is then determined whether the signal-to-noise ratio is lower than a preset threshold. Electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered, while electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

[0071] If the signal-to-noise ratio of the filtered electronic image data is lower than the preset threshold, the electronic image data is discarded, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

[0072] The process is straightforward: acquiring the target user's current and historical inspection data, including both electronic and paper-based image data. For the paper-based image data, digitization is performed, specifically involving the following steps:

[0073] (1) Compare and analyze the electronic image data and paper image data in the historical inspection data. If the electronic image data and paper image data are duplicated, use the electronic image data and delete the duplicate paper image data.

[0074] (2) Scan each of the remaining paper image data into PNG format, crop the scanned PNG images to ensure that the images occupy the full area of ​​the image, and store each image separately.

[0075] (3) Calculate the signal-to-noise ratio of each image. If the signal-to-noise ratio is lower than the preset threshold, filtering is required. If the signal-to-noise ratio of the filtered image is still lower than the preset threshold, it indicates that the original data of the "digitized" paper image is too poor and will not be used.

[0076] In this embodiment, by digitizing the paper image data and comparing it with electronic image data in historical inspection data, duplicate data can be effectively removed, reducing the storage of redundant information. By scanning all paper image data and converting it into electronic image data, and further cropping each image data and calculating the signal-to-noise ratio, the quality of the image data is effectively improved.

[0077] Step 120: Divide all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocess the homogeneous data and heterogeneous multi-source data respectively to obtain preprocessed historical inspection data.

[0078] In some embodiments, the process of dividing all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocessing the homogeneous data and heterogeneous multi-source data respectively to obtain preprocessed historical inspection data includes:

[0079] All electronic image data are compared with the parameters of the current examination data. Electronic image data with the same sensor type and resolution as the current examination data are selected as source data. The source data are numbered with 10 digits, where the first 8 digits are the image acquisition time and the last 2 digits are the block number corresponding to the three-dimensional organ represented by the image. The preprocessed source data is obtained.

[0080] Electronic image data other than data from the same source are treated as multi-source heterogeneous data, and it is determined whether the type and resolution of the multi-source heterogeneous data are the same as those of the currently inspected data.

[0081] If the multi-source heterogeneous data is of a different type than the currently inspected data, convert the multi-source heterogeneous data into data of the same type as the currently inspected data.

[0082] If the resolution of the multi-source heterogeneous data is different from that of the current inspection data, the multi-source heterogeneous data will be converted into data with the same resolution as the current inspection data.

[0083] The multi-source heterogeneous data are numbered 10 times, with the first 8 digits representing the image acquisition time and the last 2 digits representing the block number corresponding to the three-dimensional organ represented by the image, thus obtaining the preprocessed multi-source heterogeneous data.

[0084] It is easy to understand that all electronic image data are compared with the parameters of the current examination data, and electronic image data with the same sensor type (for example, considering both black and white and color cases) and the same resolution are selected as source data.

[0085] Data from the same source are numbered, for example, in the format 2023101512; where the first 8 digits indicate the time of image acquisition and the last 2 digits indicate the block number of the three-dimensional organ in which the image is located.

[0086] Electronic image data other than homogeneous data are treated as multi-source heterogeneous data. It is determined whether the type and resolution of the multi-source heterogeneous data are the same as those of the current inspection data. If the current inspection data is a black and white image, while the multi-source heterogeneous data is a pseudo-color image, the multi-source heterogeneous data is converted to grayscale space and transformed into a black and white image according to different pseudo-color schemes.

[0087] If the current inspection data is a pseudo-color image, while the multi-source heterogeneous data is a black and white image, then a generative adversarial network needs to be constructed to convert the black and white image into a pseudo-color image. The specific process is as follows:

[0088] (1) Use the Tensorflow machine learning framework to build a deep learning runtime environment;

[0089] (2) Style transfer (black and white to pseudo-color) of images is achieved by using the Pix2Pix model in adversarial generative networks.

[0090] (3) Construct a sample library containing endoscopic ultrasound pseudo-color images and their corresponding black and white images, with a sample size of 1000 pairs;

[0091] (4) Train the Pix2Pix model to obtain the final style transfer model for pseudocolor image generation;

[0092] (5) Use style transfer model to convert the black and white ultrasound endoscopic images from the past into pseudo-color images that are consistent with the current examination data.

[0093] If the resolution of the current examination data differs from that of the multi-source heterogeneous data, there are two scenarios. If the multi-source heterogeneous data has a higher resolution and the current examination data has a lower resolution, the high-resolution image of the multi-source heterogeneous data is directly converted into an image with the same resolution as the current ultrasound endoscopic examination. If the multi-source heterogeneous data has a higher resolution and the current resolution is lower, super-resolution reconstruction is required. The super-resolution reconstruction method is accomplished using a generative adversarial network model.

[0094] In this embodiment, by classifying all electronic image data and processing homogeneous data and heterogeneous multi-source data separately, the quality and consistency of the data can be effectively improved. By filtering out homogeneous data with the same sensor type and resolution as the current inspection data and assigning them numbers, the accuracy and traceability of homogeneous data are improved. By converting heterogeneous multi-source data of different types or resolutions to maintain consistency with the current inspection data, multi-source data can be effectively integrated, improving the efficiency and reliability of data processing.

[0095] Step 130: Based on the target organ corresponding to the current examination data, establish a three-dimensional model and perform mesh division to obtain a three-dimensional model including multiple meshes;

[0096] In some embodiments, the step of establishing a three-dimensional model based on the target organ corresponding to the current examination data and performing mesh division to obtain a three-dimensional model including multiple meshes includes:

[0097] Construct a three-dimensional model of the target organ corresponding to the current examination data;

[0098] The three-dimensional model is meshed to obtain multiple meshes;

[0099] Each grid is numbered. Based on the relationship between the last two digits of the preprocessed historical inspection data block number and the grid number, the preprocessed homogeneous data and multi-source heterogeneous data are applied to the surface of the 3D model corresponding to the grid as surface textures, resulting in a 3D model including multiple grids.

[0100] The process is straightforward: construct a 3D model of the target organ corresponding to the current examination data, such as a human stomach or lung model. This 3D model is then meshed in 3D space, divided into multiple cubic meshes. All meshes are numbered; for example, if divided into 64 4x4x4 meshes, the numbers range from 01 to 64. Based on historical data, image files with the same first 8 digits of their image numbers are grouped together (i.e., the same detection result is grouped together). Each group of 64 meshes is further divided into 64 subgroups. The image corresponding to each subgroup is then applied as a surface texture to the surface of the corresponding generic 3D model. This process includes the following steps:

[0101] (1) Historical images are pasted one by one onto the surface of the three-dimensional model corresponding to the grid. The operator can check the pasting accuracy through human-computer interaction.

[0102] (2) The image repeating area is processed by pixel averaging. If a pixel in a certain area has three images superimposed, the three pixel values ​​of that point are averaged.

[0103] (3) If the images of the overlapping regions are significantly different, such as the differences caused by organ peristalsis, the images with significant differences should be discarded and the pixel average of the similar images should be taken directly. If there are only two images of the overlapping region and they are significantly different, one image should be discarded at random. The differences in the images can be calculated using the structural similarity parameter.

[0104] In this embodiment, by constructing a 3D model of the target organ and performing meshing, each mesh is numbered. The relationship between the last two digits of the preprocessed historical examination data's block number and the mesh number is used to apply homogeneous and heterogeneous data from multiple sources to the surface of the 3D model as surface textures. This effectively improves the accuracy of the 3D model and the data integration effect. It enables a more accurate combination of historical data and current examination data, achieving 3D simulation construction of the organ corresponding to the examination data.

[0105] Step 140: Input the preprocessed historical inspection data into the 3D model, assign weights to each grid in the 3D model, and obtain the target 3D model;

[0106] In some embodiments, the step of inputting preprocessed historical inspection data into a 3D model, assigning weights to each mesh in the 3D model, and obtaining a target 3D model includes:

[0107] The preprocessed historical inspection data is input into the 3D model, and the historical inspection data of each grid is numbered.

[0108] Calculate the time interval between the historical inspection data and the current inspection data for each grid;

[0109] The historical inspection data number for each grid is updated based on the aforementioned time interval;

[0110] A weighting formula is constructed based on the updated historical inspection data numbering, and weights are assigned to each grid in the 3D model to obtain the target 3D model.

[0111] It should be noted that, considering that historical examinations may not always yield complete observational data of the examined organ (they may only involve partial examinations, resulting in localized images), it is necessary to assign appropriate weights to data from different periods and of different quality. This includes the following steps:

[0112] (1) For each of the 64 grids, collect information on whether the grid contains historical detection images;

[0113] (2) Number the historical data of each grid, such as MB101, which represents the data of the first grid and the time point closest to the current detection; and MB102, which represents the historical monitoring data of the second time before the current detection.

[0114] (3) Calculate the time interval between the historical data of each grid and the current time. Considering that diseased organs are usually re-examined every 3 months or half a year, the data is marked in quarterly units.

[0115] (4) Update the grid historical data number in (2), such as MB1026, which represents the second historical monitoring data before the current detection, and the time interval between it and the current is 6 cycle units (each cycle unit is 3 months), that is, an interval of 18 months;

[0116] (5) Set a threshold of 20, which means that if the time interval between historical data and the current data exceeds the threshold of 20 periodic units (5 years), the previous data will not be used;

[0117] (6) Construct the weight allocation formula:

[0118]

[0119] Where i is the grid number; j represents the j-th backward detection data; and m represents the period unit between the current detection and the current detection. Let be the weight of the data from the j-th detection.

[0120] In some embodiments, such as grid number 11, there are 3 historical data points, spaced at intervals of 2, 6, and 12 periodic units respectively. The weights of these 3 historical data points for that grid can be expressed by the formula:

[0121]

[0122]

[0123]

[0124] in, The weights for the first historical data of grid number 11, The weights for the second historical data of grid number 11, The weight of the third historical data for grid number 11.

[0125] In this embodiment, by inputting preprocessed historical inspection data into the 3D model and numbering and calculating time intervals for each grid, the time difference between each grid and the current inspection data can be more accurately reflected. Based on the updated historical inspection data numbers, a weight allocation formula is constructed and weights are allocated, enabling auxiliary detection of historical data in local areas. By considering the time factor, the processing of historical data is more timely, and the impact of data in each grid on the current inspection data can be better evaluated.

[0126] Step 150: Generate predictive inspection data based on the target 3D model, compare and optimize the current inspection data to obtain the target inspection data.

[0127] In some embodiments, the step of generating predictive inspection data based on the target 3D model and comparing and optimizing the current inspection data to obtain the target inspection data includes:

[0128] A time-series 3D visualization model of multiple local parts is established based on the target 3D model;

[0129] Predictive inspection data is generated based on a time-series 3D visualization model of multiple local parts. The current inspection data is then compared and optimized to obtain the target inspection data.

[0130] In some embodiments, the step of generating predictive inspection data based on a temporal 3D visualization model of multiple local parts, comparing and optimizing the current inspection data to obtain target inspection data includes:

[0131] Rotate the time-series 3D visualization models of multiple local parts to the same viewpoint and capture multiple images;

[0132] Weights are assigned to multiple images based on a weighting formula to generate predictive inspection data.

[0133] Calculate the mean square error and structural similarity index between the predicted inspection data and the current inspection data, compare and optimize the current inspection data to obtain the target inspection data.

[0134] The concept is easy to understand: preprocessed historical data and the generated target 3D model are used to achieve temporal 3D observation of the current results. Simultaneously, based on the historical results of corresponding parts (mesh), the current detection is optimized. First, a temporal 3D visualization model of the local area is established for more intuitive comparative observation. The specific process is as follows:

[0135] (1) Observe which grids are included in the local area and record the grid numbers;

[0136] (2) For each grid, determine whether it has historical detection data;

[0137] (3) If historical detection data exists, generate a historical 3D image of the grid.

[0138] (4) Render the historical 3D images of the mesh onto the same interface for display;

[0139] (5) Operation functions of historical 3D images bound to the grid, that is, when any historical 3D model is operated (enlarged, reduced, rotated), other historical 3D models in the same grid will change at the same rate and scale, which is convenient for real-time observation and diagnosis.

[0140] Optionally, the SSIM model can be used to quantify the structural differences in test results at different times to help assess disease progression.

[0141] Predictive inspection data is generated based on a time-series 3D visualization model of multiple local parts. The current inspection data is then compared and optimized to obtain the target inspection data. The specific process is as follows:

[0142] (1) If there are at least 3 historical detection results for a certain grid area being detected, the SSIM value of the historical detection results (grid) is tested first; the detection method is to select 5 arbitrary viewpoints for the grid and capture two-dimensional images.

[0143] (2) Calculate SSIM pairwise between the captured 2D image and the image with the same viewpoint in the historical detection data;

[0144] (3) If all SSIM calculation results are greater than the preset threshold (e.g., 0.8), it indicates that the selected images have high similarity (strong image consistency, which is convenient for subsequent image generation). Then, it is marked that the historical data of the entire grid has high similarity, which meets the requirements for subsequent image generation.

[0145] (4) For the current real-time detection, when a certain angle is selected for shooting, the shooting result is recorded as pcurrent. In the grid corresponding to that angle, all historical grids of all periods are rotated to the same viewpoint and the images are captured. The images are recorded as p1, p2, ... pn, where n represents the observation data of n historical periods.

[0146] (5) Assign weights to the data p1, p2, ... pn and generate a predicted image pfake;

[0147] (6) Calculate the SSIM and MSE of the two images, pcurrent and pfake, which can comprehensively and quantitatively evaluate the difference between the predicted and actual results of the organ regions corresponding to the images, and assist in detection.

[0148] In this embodiment, by rotating the temporal 3D visualization models of multiple local parts to the same viewpoint and capturing multiple images, and then weighting these images based on a weighting formula, more accurate predictive inspection data can be generated. By calculating the mean square error and structural similarity index between the predicted inspection data and the current inspection data, the current inspection data is compared and optimized to obtain the target inspection data. This improves the accuracy of the inspection data and the efficiency of the comparison and optimization, thereby enhancing the accuracy and authenticity of the inspection data.

[0149] In this embodiment, by establishing time-series 3D visualization models of multiple local parts and generating predictive inspection data based on these models, the current inspection data can be efficiently compared and optimized. By binding and combining historical 3D models with the current inspection data, real-time 3D comparative observation is achieved, which facilitates efficient decision support based on the current inspection data.

[0150] The data optimization method based on multi-source heterogeneous time-series assistance provided in this application obtains the current and historical examination data of the target user and digitizes paper image data into electronic image data. This effectively integrates homogeneous data and multi-source heterogeneous data, improving data quality and consistency. A three-dimensional model is established based on the target organ corresponding to the current examination data, and the model is meshed and weighted. By inputting historical data into the three-dimensional model to generate predicted examination data and comparing and optimizing it with the current examination data, the accuracy and authenticity of the examination data are improved.

[0151] The data optimization method based on multi-source heterogeneous timing assistance provided in this application can be executed by a data optimization device based on multi-source heterogeneous timing assistance. This application uses the execution of the data optimization method based on multi-source heterogeneous timing assistance by the data optimization device as an example to illustrate the data optimization device based on multi-source heterogeneous timing assistance provided in this application.

[0152] This application also provides a data optimization device based on multi-source heterogeneous timing assistance, such as... Figure 2 As shown, the data optimization device based on multi-source heterogeneous timing assistance includes: an acquisition module 210, a first processing module 220, a second processing module 230, a third processing module 240, and an optimization module 250.

[0153] The acquisition module 210 is used to acquire the current inspection data and historical inspection data of the target user. The historical inspection data includes electronic image data and paper image data. The paper image data is converted into electronic image data through image digitization processing.

[0154] The first processing module 220 is used to divide all electronic image data into homogeneous data and multi-source heterogeneous data, and preprocess the homogeneous data and multi-source heterogeneous data respectively to obtain preprocessed historical inspection data.

[0155] The second processing module 230 is used to build a three-dimensional model based on the target organ corresponding to the current examination data and perform mesh division to obtain a three-dimensional model including multiple meshes.

[0156] The third processing module 240 is used to input the preprocessed historical inspection data into the three-dimensional model, assign weights to each grid in the three-dimensional model, and obtain the target three-dimensional model.

[0157] The optimization module 250 is used to generate predictive inspection data based on the target 3D model, compare and optimize the current inspection data, and obtain the target inspection data.

[0158] The data optimization method based on multi-source heterogeneous time-series assistance provided in this application obtains the current and historical examination data of the target user and digitizes paper image data into electronic image data. This effectively integrates homogeneous data and multi-source heterogeneous data, improving data quality and consistency. A three-dimensional model is established based on the target organ corresponding to the current examination data, and the model is meshed and weighted. By inputting historical data into the three-dimensional model to generate predicted examination data and comparing and optimizing it with the current examination data, the accuracy and authenticity of the examination data are improved.

[0159] The data optimization device based on multi-source heterogeneous time-assisted optimization provided in this application embodiment can achieve... Figure 1 The various processes implemented in the data optimization method based on multi-source heterogeneous time-series assistance will not be described in detail here to avoid repetition.

[0160] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-described data optimization method embodiment based on multi-source heterogeneous timing assistance and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0161] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0162] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described data optimization method embodiment based on multi-source heterogeneous timing assistance and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0163] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0164] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described data optimization method based on multi-source heterogeneous timing assistance.

[0165] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0166] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described data optimization method embodiment based on multi-source heterogeneous timing assistance, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0167] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a device-level chip, device chip, chip device, or on-chip device chip, etc.

[0168] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0169] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the data optimization method based on multi-source heterogeneous timing assistance of the various embodiments of this application.

[0170] In the description of this application, "first feature" and "second feature" may include one or more of the features.

[0171] In the description of this application, "multiple" means two or more.

[0172] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0173] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0174] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A data optimization method based on multi-source heterogeneous time-series assistance, characterized in that, The method includes: Obtain the target user's current inspection data and historical inspection data. The historical inspection data includes electronic image data and paper image data. Convert the paper image data into electronic image data through image digitization. All electronic image data are divided into homogeneous data and heterogeneous multi-source data. The homogeneous data and heterogeneous multi-source data are preprocessed separately to obtain preprocessed historical inspection data. A three-dimensional model is built based on the target organ corresponding to the current examination data and meshed to obtain a three-dimensional model including multiple meshes; The preprocessed historical inspection data is input into the 3D model, and the historical inspection data of each grid in the 3D model is weighted to obtain the target 3D model. Predictive inspection data is generated based on the target 3D model, and the current inspection data is compared and optimized to obtain the target inspection data. The process of establishing a three-dimensional model of the target organ based on the current examination data and performing mesh generation yields a three-dimensional model comprising multiple meshes, including: Construct a three-dimensional model of the target organ corresponding to the current examination data; The three-dimensional model is meshed to obtain multiple meshes; Each grid is numbered. Based on the relationship between the last two digits of the preprocessed historical inspection data block number and the grid number, the preprocessed homogeneous data and multi-source heterogeneous data are applied to the surface of the 3D model corresponding to the grid in the form of surface texture, so as to obtain a 3D model including multiple grids. The process of inputting preprocessed historical inspection data into a 3D model, assigning weights to the historical inspection data of each grid in the 3D model, and obtaining the target 3D model includes: The preprocessed historical inspection data is input into the 3D model, and the historical inspection data of each grid is numbered. Calculate the time interval between the historical inspection data and the current inspection data for each grid; The historical inspection data number for each grid is updated based on the aforementioned time interval; A weighting formula is constructed based on the updated historical inspection data numbers. Weights are then assigned to the historical inspection data of each grid in the 3D model to obtain the target 3D model.

2. The data optimization method based on multi-source heterogeneous time-series assistance according to claim 1, characterized in that, The process of converting paper image data into electronic image data includes: Determine whether electronic image data and paper image data in historical inspection data are duplicates. If electronic image data and paper image data are duplicates, delete the paper image data that is duplicated with the electronic image data. If electronic image data and paper image data are not duplicates, retain both electronic image data and paper image data. All paper image data are scanned to obtain electronic image data in PNG format; After cropping each PNG format electronic image data, the signal-to-noise ratio is calculated. It is then determined whether the signal-to-noise ratio is lower than a preset threshold. Electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered, while electronic image data with a signal-to-noise ratio higher than the preset threshold is retained. If the signal-to-noise ratio of the filtered electronic image data is lower than the preset threshold, the electronic image data is discarded, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

3. The data optimization method based on multi-source heterogeneous time-series assistance according to claim 1, characterized in that, The process involves dividing all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocessing both types of data to obtain preprocessed historical inspection data, including: All electronic image data are compared with the parameters of the current examination data. Electronic image data with the same sensor type and resolution as the current examination data are selected as source data. The source data are numbered with 10 digits, where the first 8 digits are the image acquisition time and the last 2 digits are the block number corresponding to the three-dimensional organ represented by the image. The preprocessed source data is obtained. Electronic image data other than data from the same source are treated as multi-source heterogeneous data, and it is determined whether the type and resolution of the multi-source heterogeneous data are the same as those of the currently inspected data. If the multi-source heterogeneous data is of a different type than the currently inspected data, convert the multi-source heterogeneous data into data of the same type as the currently inspected data. If the resolution of the multi-source heterogeneous data is different from that of the current inspection data, the multi-source heterogeneous data will be converted into data with the same resolution as the current inspection data. The multi-source heterogeneous data are numbered 10 times, with the first 8 digits representing the image acquisition time and the last 2 digits representing the block number corresponding to the three-dimensional organ represented by the image, thus obtaining the preprocessed multi-source heterogeneous data.

4. The data optimization method based on multi-source heterogeneous time-series assistance according to claim 1, characterized in that, The process of generating predictive inspection data based on the target 3D model, comparing and optimizing the current inspection data to obtain the target inspection data includes: A time-series 3D visualization model of multiple local parts is established based on the target 3D model; Predictive inspection data is generated based on a time-series 3D visualization model of multiple local parts. The current inspection data is then compared and optimized to obtain the target inspection data.

5. The data optimization method based on multi-source heterogeneous time-series assistance according to claim 4, characterized in that, The predictive inspection data is generated based on a time-series 3D visualization model of multiple local parts. The current inspection data is then compared and optimized to obtain the target inspection data, including: Rotate the time-series 3D visualization models of multiple local parts to the same viewpoint and capture multiple images; Weights are assigned to multiple images based on a weighting formula to generate predictive inspection data. Calculate the mean square error and structural similarity index between the predicted inspection data and the current inspection data, compare and optimize the current inspection data to obtain the target inspection data.

6. A data optimization device based on multi-source heterogeneous time-series assistance, implemented using the data optimization method based on multi-source heterogeneous time-series assistance as described in any one of claims 1 to 5, characterized in that, The device includes: The acquisition module is used to acquire the target user's current inspection data and historical inspection data. The historical inspection data includes electronic image data and paper image data. The paper image data is converted into electronic image data through image digitization processing. The first processing module is used to divide all electronic image data into homogeneous data and heterogeneous multi-source data, and preprocess the homogeneous data and heterogeneous multi-source data respectively to obtain preprocessed historical inspection data. The second processing module is used to build a three-dimensional model of the target organ corresponding to the current examination data and perform mesh division to obtain a three-dimensional model including multiple meshes. The third processing module is used to input the preprocessed historical inspection data into the 3D model, assign weights to the historical inspection data of each grid in the 3D model, and obtain the target 3D model. The optimization module is used to generate predictive inspection data based on the target 3D model, compare and optimize the current inspection data to obtain the target inspection data.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data optimization method based on multi-source heterogeneous timing assistance as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data optimization method based on multi-source heterogeneous timing assistance as described in any one of claims 1 to 5.

Images