Data optimization method and device based on multi-source heterogeneous time sequence assistance

Abstract

The invention discloses a data optimization method and device based on multi-source heterogeneous time sequence assistance, and belongs to the technical field of data optimization. The method comprises the following steps: acquiring current inspection data and historical inspection data of a target user, and performing image electronization processing on paper image data to convert the paper image data into electronic image data; dividing all the electronic image data into homologous data and multi-source heterogeneous data, and preprocessing the homologous data and the multi-source heterogeneous data to obtain preprocessed historical check data; establishing a three-dimensional model based on a target organ corresponding to the current examination data, and performing grid division to obtain a three-dimensional model comprising a plurality of grids; inputting the preprocessed historical inspection data into the three-dimensional model, and carrying out weight distribution on each grid in the three-dimensional model to obtain a target three-dimensional model; and generating prediction inspection data based on the target three-dimensional model, and comparing and optimizing the current inspection data to obtain target inspection data. The method improves the accuracy and authenticity of the data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data optimization, and particularly relates to a data optimization method and device based on multi-source heterogeneous time sequence assistance. BACKGROUND

[0002] An ultrasonic endoscope is a medical device with continuously improving technology and expanding functions, which fills some special indications that cannot be covered by ordinary endoscopes, body surface ultrasound and CT devices. In recent years, with the continuous development of ultrasonic endoscope technology, its application in the medical field has become more and more widespread, becoming an important means of checking various diseases. However, the current ultrasonic endoscopic examination still faces many challenges, especially in terms of image quality and data processing.

[0003] The existing ultrasonic endoscope technology is limited by the size of the endoscope and the performance of the ultrasonic sensor, resulting in low resolution and more noise in real-time images, which brings certain difficulties to diagnosis and evaluation. Since the size of the endoscope needs to be strictly controlled and the performance of the ultrasonic sensor has not yet reached the ideal state, the clarity and detail performance of the ultrasonic endoscope image in actual application are often not as good as other imaging devices, affecting the accurate judgment of the doctor on the lesion. In addition, the ultrasonic endoscope mainly enters the target area through the natural cavity of the human body for local observation. This local observation method is difficult to construct a real-time three-dimensional model of the organ in the same dimension, thereby affecting the evaluation of the overall state of the organ. Although the ultrasonic endoscope can observe the target organ in detail, its limitation is that it can only provide two-dimensional images and cannot provide complete three-dimensional spatial information, which poses a challenge to the diagnosis and treatment plan of complex lesions.

[0004] The existing technology may cause certain differences in the results of each examination due to differences in equipment, operating personnel and environment during ultrasonic endoscope examination, which cannot be efficiently compared longitudinally. This inconsistency not only increases the burden of examination, but also affects the continuity and accuracy of diagnosis. The existing examination data has the problems of low accuracy and continuity. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes a data optimization method based on multi-source heterogeneous time sequence assistance, which improves the accuracy and authenticity of the data.

[0006] In a first aspect, the present application provides a data optimization method based on multi-source heterogeneous time sequence assistance, which comprises: obtaining current examination data and historical examination data of a target user, the historical examination data comprising electronic image data and paper image data, and converting the paper image data into electronic image data through image electronic processing; All electronic image data is divided into homologous data and multi-source heterogeneous data, and the homologous data and the multi-source heterogeneous data are respectively preprocessed to obtain preprocessed historical examination data; A three-dimensional model is established based on a target organ corresponding to the current examination data and is meshed to obtain a three-dimensional model including a plurality of meshes; The preprocessed historical examination data is input into the three-dimensional model, and weight distribution is performed on each mesh in the three-dimensional model to obtain a target three-dimensional model; The target three-dimensional model is used to generate predicted examination data, and the current examination data is compared and optimized to obtain target examination data.

[0007] According to an embodiment of the present application, the paper image data is converted into electronic image data through image electronic processing, which comprises: It is judged whether the electronic image data and the paper image data in the historical examination data are repeated, if the electronic image data and the paper image data are repeated, the paper image data repeated with the electronic image data is deleted, if the electronic image data and the paper image data are not repeated, the electronic image data and the paper image data are retained; All paper image data is scanned to obtain png format electronic image data; After each png format electronic image data is cropped, the signal-to-noise ratio is calculated, it is judged whether the signal-to-noise ratio is lower than a preset threshold, the electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained; If the signal-to-noise ratio of the filtered electronic image data is lower than the preset threshold, the electronic image data is discarded, and the electronic image data with a signal-to-noise ratio higher than the preset threshold after filtering is retained.

[0008] According to an embodiment of the present application, all electronic image data is divided into homologous data and multi-source heterogeneous data, and the homologous data and the multi-source heterogeneous data are respectively preprocessed to obtain preprocessed historical examination data, which comprises: All electronic image data is compared with the parameters of the current examination data, and electronic image data consistent with the sensor type and resolution of the current examination data is selected as homologous data, the homologous data is numbered, the number of numbers is 10, the first 8 bits are image acquisition time, and the last 2 bits are the block number corresponding to the three-dimensional organ represented by the image, to obtain preprocessed homologous data; The electronic image data other than the homologous data is used as multi-source heterogeneous data, and it is judged whether the multi-source heterogeneous data is the same as the type and resolution of the current examination data; If the type of the multi-source heterogeneous data is different from that of the current examination data, the multi-source heterogeneous data is converted into data of the same type as the current examination data; If the multi-source heterogeneous data has different resolution from the current examination data, the multi-source heterogeneous data is converted into data with the same resolution as the current examination data. The multi-source heterogeneous data is numbered, and the number of the numbering is 10, where the first 8 bits are the image acquisition time, and the last 2 bits are the block number corresponding to the three-dimensional organ represented by the image, to obtain the pre-processed multi-source heterogeneous data.

[0009] According to an embodiment of the present application, the three-dimensional model corresponding to the target organ of the current examination data is established and meshed to obtain a three-dimensional model including a plurality of meshes, including: constructing a three-dimensional model of the target organ corresponding to the current examination data; meshing the three-dimensional model to obtain a plurality of meshes; numbering each mesh, and based on the relationship between the last two bits of the block number of the pre-processed historical examination data and the mesh number, the pre-processed homologous data and the multi-source heterogeneous data are pasted to the surface of the three-dimensional model corresponding to the mesh in the form of surface texture to obtain a three-dimensional model including a plurality of meshes.

[0010] According to an embodiment of the present application, the pre-processed historical examination data is input into the three-dimensional model, and each mesh in the three-dimensional model is assigned a weight to obtain a target three-dimensional model, including: inputting the pre-processed historical examination data into the three-dimensional model, and numbering the historical examination data of each mesh; calculating the time interval of each mesh historical examination data from the current examination data; updating the historical examination data number of each mesh based on the time interval; constructing a weight assignment formula based on the updated historical examination data number, and assigning a weight to each mesh in the three-dimensional model to obtain a target three-dimensional model.

[0011] According to an embodiment of the present application, the pre-processed historical examination data is input into the three-dimensional model, and each mesh in the three-dimensional model is assigned a weight to obtain a target three-dimensional model, including: establishing a time sequence three-dimensional visualization model of a plurality of local parts based on the target three-dimensional model; generating prediction examination data based on the time sequence three-dimensional visualization model of a plurality of local parts, and comparing and optimizing the current examination data to obtain target examination data.

[0012] According to an embodiment of the present application, the pre-processed historical examination data is input into the three-dimensional model, and each mesh in the three-dimensional model is assigned a weight to obtain a target three-dimensional model, including: The time sequence three-dimensional visualization models of the plurality of local parts are all rotated to the same perspective, and a plurality of images are intercepted; The plurality of images are assigned weights based on a weight assignment formula, and prediction inspection data is generated; The mean square error and the structural similarity index of the prediction inspection data and the current inspection data are calculated, the current inspection data is compared and optimized, and target inspection data is obtained.

[0013] In a second aspect, the application provides a data optimization device based on multi-source heterogeneous time sequence assistance, which comprises: An acquisition module is configured to acquire target user current inspection data and historical inspection data, wherein the historical inspection data comprises electronic image data and paper image data, and the paper image data is converted into electronic image data through image electronic processing; A first processing module is configured to divide all the electronic image data into homogeneous data and multi-source heterogeneous data, and pre-process the homogeneous data and the multi-source heterogeneous data respectively to obtain pre-processed historical inspection data; A second processing module is configured to establish a three-dimensional model based on a target organ corresponding to the current inspection data and perform grid division to obtain a three-dimensional model comprising a plurality of grids; A third processing module is configured to input the pre-processed historical inspection data into the three-dimensional model, and perform weight assignment on each grid in the three-dimensional model to obtain a target three-dimensional model; An optimization module is configured to generate prediction inspection data based on the target three-dimensional model, compare and optimize the current inspection data, and obtain target inspection data.

[0014] In a third aspect, the application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data optimization method based on multi-source heterogeneous time sequence assistance according to the first aspect.

[0015] In a fourth aspect, the application provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the data optimization method based on multi-source heterogeneous time sequence assistance according to the first aspect.

[0016] In a fifth aspect, the application provides a chip, which comprises a processor and a communication interface, the communication interface and the processor are coupled, and the processor is configured to run a program or an instruction to implement the data optimization method based on multi-source heterogeneous time sequence assistance according to the first aspect.

[0017] In a sixth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the multi-source heterogeneous timing-aided data optimization method according to the first aspect above.

[0018] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings.

[0019] The multi-source heterogeneous timing-aided data optimization method provided by the present application has the following beneficial effects compared with the prior art: (1) The present application can effectively integrate homogenous data and multi-source heterogeneous data by obtaining current examination data and historical examination data of a target user and electronically processing paper image data into electronic image data, thereby improving the quality and consistency of the data. A three-dimensional model is established based on the target organ corresponding to the current examination data, and the model is meshed and weighted. The historical data is input into the three-dimensional model to generate predicted examination data and compared with the current examination data for optimization, thereby improving the accuracy and authenticity of the examination data.

[0020] (2) The present application can effectively improve the quality and consistency of the data by classifying all electronic image data and processing homogenous data and multi-source heterogeneous data respectively. The homogenous data with the same sensor type and resolution as the current examination data is screened out and numbered, thereby improving the accuracy and traceability of the homogenous data. The multi-source heterogeneous data of different types or resolutions is converted to maintain consistency with the current examination data, thereby effectively integrating multi-source data and improving the efficiency and reliability of data processing.

[0021] (3) The present application can more accurately reflect the time difference between each grid and the current examination data by inputting the preprocessed historical examination data into the three-dimensional model and numbering each grid and calculating the time interval. Based on the updated historical examination data numbering, a weight distribution formula is constructed and weighted, thereby realizing the auxiliary detection of local area historical data. The processing of historical data is more time-efficient by considering the time factor, thereby better evaluating the influence of data in each grid on the current examination data. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which: Figure 1 is a flowchart of the multi-source heterogeneous timing-aided data optimization method provided by the embodiments of the present application; Figure 2is a structural schematic diagram of a data optimization device based on multi-source heterogeneous timing assistance provided by an embodiment of the present application. Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0024] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the front and rear associated objects are in an "or" relationship.

[0025] The data optimization method based on multi-source heterogeneous timing assistance, the data optimization device based on multi-source heterogeneous timing assistance, the electronic device and the readable storage medium provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and their application scenarios.

[0026] The data optimization method based on multi-source heterogeneous timing assistance can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0027] The terminal includes but is not limited to a portable communication device such as a mobile phone or a tablet computer having a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad). It should also be understood that in some embodiments, the terminal can not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or a touchpad).

[0028] In each of the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal can include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0029] The method for optimizing data based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application can be executed by an electronic device or a functional module or functional entity capable of implementing the method for optimizing data based on multi-source heterogeneous time sequence assistance in the electronic device. The electronic device mentioned in the embodiments of the present application includes but is not limited to a mobile phone, a tablet computer, a computer, a camera, a wearable device, and the like. The method for optimizing data based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application will be described below by taking an electronic device as an example.

[0030] Figure 1 FIG. 1 is a flowchart of the method for optimizing data based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application. As shown in FIG. 1, the method for optimizing data based on multi-source heterogeneous time sequence assistance includes steps 110, 120, 130, 140, and 150. Figure 1

[0031] In step 110, current examination data and historical examination data of a target user are acquired. The historical examination data includes electronic image data and paper image data. The paper image data is converted into electronic image data through image electronicization processing. In some embodiments, the conversion of the paper image data into electronic image data through image electronicization processing includes: In step 120, it is determined whether the electronic image data and the paper image data in the historical examination data are repeated. If the electronic image data and the paper image data are repeated, the paper image data that is repeated with the electronic image data is deleted. If the electronic image data and the paper image data are not repeated, the electronic image data and the paper image data are retained. In step 130, all the paper image data is scanned to obtain electronic image data in png format. In step 140, each piece of electronic image data in png format is cropped and the signal-to-noise ratio is calculated. It is determined whether the signal-to-noise ratio is lower than a preset threshold. The electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered. The electronic image data with a signal-to-noise ratio higher than the preset threshold is retained. In step 150, if the signal-to-noise ratio of the filtered electronic image data is lower than the preset threshold, the electronic image data is discarded. The electronic image data with a signal-to-noise ratio higher than the preset threshold after filtering is retained.

[0032] It is easy to understand that the current examination data and the historical examination data of a target user are acquired. The historical examination data includes electronic image data and paper image data. The paper image data is processed through image electronicization. The specific process includes the following steps: (1) The electronic image data and the paper image data in the historical examination data are compared and analyzed. If the electronic image data and the paper image data are repeated, the electronic image data is used and the repeated paper image data is deleted. ​

[0033] (2) The remaining paper image data is scanned one by one into png format, and the scanned png image is cropped to ensure that the image occupies the full width of the image, and each image is stored separately.

[0034] (3) Calculate the signal-to-noise ratio of each image. If the signal-to-noise ratio is lower than the preset threshold, filtering is needed. If the signal-to-noise ratio of the filtered image is still lower than the preset threshold, it means that the original data of the paper image after "electronic" is too poor, and it is not used.

[0035] In this embodiment, by electronic processing of paper image data and comparing with electronic image data in historical inspection data, repeated data can be effectively removed and redundant information storage can be reduced. By scanning all paper image data and converting it into electronic image data, further cropping each image data and calculating the signal-to-noise ratio, the quality of image data is effectively improved.

[0036] Step 120, divide all electronic image data into homologous data and multi-source heterogeneous data, respectively preprocess homologous data and multi-source heterogeneous data to obtain preprocessed historical inspection data; In some embodiments, the electronic image data is divided into homologous data and multi-source heterogeneous data, and the homologous data and multi-source heterogeneous data are preprocessed to obtain preprocessed historical inspection data, comprising: Compare all electronic image data with the parameters of the current inspection data, and select electronic image data consistent with the sensor type and resolution of the current inspection data as homologous data. Number the homologous data, the number of which is 10, the first 8 bits are the image acquisition time, and the last 2 bits are the block number corresponding to the three-dimensional organ represented by the image. Get preprocessed homologous data; The electronic image data other than the homologous data is used as multi-source heterogeneous data, and it is judged whether the multi-source heterogeneous data is the same as the type and resolution of the current inspection data; If the type of the multi-source heterogeneous data is different from the current inspection data, the multi-source heterogeneous data is converted into data of the same type as the current inspection data; If the resolution of the multi-source heterogeneous data is different from the current inspection data, the multi-source heterogeneous data is converted into data of the same resolution as the current inspection data; Number the multi-source heterogeneous data, the number of which is 10, the first 8 bits are the image acquisition time, and the last 2 bits are the block number corresponding to the three-dimensional organ represented by the image. Get preprocessed multi-source heterogeneous data.

[0037] It is easy to understand that all electronic image data is compared with the parameters of the current examination data, and the electronic image data consistent with the current sensor type (for example, considering both black and white and color) and resolution is screened out as homologous data.

[0038] The homologous data is numbered, and the numbering format is, for example, 2023101512; wherein the first 8 bits indicate the image acquisition time, and the last 2 bits indicate the block number corresponding to the three-dimensional organ where the image is located.

[0039] The electronic image data other than the homologous data is taken as multi-source heterogeneous data, and it is judged whether the multi-source heterogeneous data is the same as the type and resolution of the current examination data. If the current examination data is a black and white image, and the multi-source heterogeneous data is a pseudo-color image, the multi-source heterogeneous data is converted to a gray space according to different pseudo-color matching schemes, and is converted to a black and white image.

[0040] If the current examination data is a pseudo-color image, and the multi-source heterogeneous data is a black and white image, a generative adversarial network needs to be constructed to convert the black and white image to a pseudo-color image, and the specific process is as follows: (1) Use the Tensorflow machine learning framework to build a deep learning running environment; (2) Use the Pix2Pix model in the generative adversarial network to realize the style transfer of the image (black and white to pseudo-color); (3) Build a sample library, which contains ultrasonic endoscopic pseudo-color images and corresponding black and white images, and the sample size is 1000 pairs; (4) Train the Pix2Pix model to obtain the final style transfer model for pseudo-color image generation; (5) Use the style transfer model to convert the past ultrasonic endoscopic black and white images to pseudo-color images consistent with the current examination data.

[0041] If the resolution of the current examination data and the multi-source heterogeneous data is different, there are also two cases. If the resolution of the multi-source heterogeneous data is higher, and the resolution of the current examination data is lower, the high-resolution image of the multi-source heterogeneous data is directly converted to the same resolution image as the current ultrasonic endoscopic detection. If the resolution of the multi-source heterogeneous data is higher, and the resolution is lower, super-resolution reconstruction needs to be done, and the super-resolution reconstruction method is completed through the generative adversarial network model.

[0042] In this embodiment, by classifying all electronic image data and processing homologous data and multi-source heterogeneous data respectively, the quality and consistency of the data can be effectively improved. By screening the homologous data consistent with the sensor type and resolution of the current examination data and performing numbering processing, the accuracy and traceability of the homologous data are improved. By converting multi-source heterogeneous data of different types or resolutions to maintain consistency with the current examination data, multi-source data can be effectively integrated, and the efficiency and reliability of data processing can be improved.

[0043] Step 130, based on the target organ corresponding to the current examination data, a three-dimensional model is established and meshed, and a three-dimensional model including a plurality of meshes is obtained; In some embodiments, the three-dimensional model corresponding to the target organ of the current examination data is established and meshed, and a three-dimensional model including a plurality of meshes is obtained, including: Constructing a three-dimensional model of the target organ corresponding to the current examination data; Grid processing the three-dimensional model to obtain a plurality of meshes; Numbering each mesh, and according to the relationship between the block number and the mesh number of the last two digits of the preprocessed historical examination data, the preprocessed homologous data and multi-source heterogeneous data are pasted to the surface of the three-dimensional model corresponding to the mesh in the form of surface texture, to obtain a three-dimensional model including a plurality of meshes.

[0044] It is easy to understand that a three-dimensional model of the target organ corresponding to the current examination data, such as a human stomach model, a lung model, etc., is meshed in a three-dimensional space and divided into a plurality of cubic meshes. All meshes are numbered, such as 64 meshes divided into 4x4x4, numbered from 01 to 64. According to the numbering of the historical data, image files with the same first 8 digits of the image number are divided into a group, i.e. the same detection result is divided into a group. The corresponding 64 meshes in each group are divided into groups, i.e. 64 groups. The image corresponding to each group is pasted to the surface of the general three-dimensional model corresponding to the mesh in the form of surface texture, which includes the following steps: (1) Paste the historical images one by one to the surface of the three-dimensional model corresponding to the mesh, which can be done by human-computer interaction and checked by the operator for accuracy; (2) The repeated area of the image is processed by pixel averaging, such as three images superimposed in a certain area, and the average of the 3 pixel values of the point is taken; (3) If the repeated area of the image differs greatly, such as the image difference caused by organ peristalsis, the image with large difference is discarded, and the pixel average of the similar image is directly taken. If there are only two images in the repeated area and the difference is large, one of them is discarded at random. The image difference can be calculated by structure similarity parameter.

[0045] In this embodiment, by constructing a three-dimensional model of the target organ and performing meshing processing, each mesh is numbered, and the relationship between the two-bit block number and the mesh number of the preprocessed historical examination data is combined. The homologous data and multi-source heterogeneous data are pasted to the surface of the three-dimensional model in the form of surface texture, effectively improving the accuracy of the three-dimensional model and the data integration effect. The historical data and the current examination data can be more accurately combined, and the three-dimensional simulation construction of the organ corresponding to the examination data is realized.

[0046] Step 140, inputting the preprocessed historical examination data into the three-dimensional model, and performing weight distribution on each mesh in the three-dimensional model to obtain a target three-dimensional model; In some embodiments, the preprocessed historical examination data is input into the three-dimensional model, and weight distribution is performed on each mesh in the three-dimensional model to obtain a target three-dimensional model, including: The preprocessed historical examination data is input into the three-dimensional model, and the historical examination data of each mesh is numbered; The time interval of each mesh historical examination data from the current examination data is calculated; The historical examination data number of each mesh is updated based on the time interval; A weight distribution formula is constructed based on the updated historical examination data number, and weight distribution is performed on each mesh in the three-dimensional model to obtain a target three-dimensional model.

[0047] It should be noted that, considering that historical detection may not always be able to obtain complete observation data of the detected organ (may only be partial detection of the organ, and the obtained is partial image of the organ), reasonable weight needs to be allocated for different periods and different quality data, including the following steps: (1) For each of the 64 meshes, collect whether there is historical detection image of the mesh; (2) Number the historical data of each mesh, such as MB101, indicating the first mesh, the data closest to the current detection; and MB102, indicating the second historical monitoring data backward from the current detection; (3) Calculate the time interval of each mesh historical data from the current time, considering that the diseased organ is usually reviewed every 3 months or every half year, so the interval is marked in quarter units; (4) Update the mesh historical data number in (2), such as MB1026, indicating the second historical monitoring data backward from the current detection, and the interval is 6 period units (3 months per period unit), i.e. 18 months; (5) Set a threshold of 20, indicating that when the interval between the historical data and the current time exceeds the threshold of 20 period units (5 years), the data before that is not used. (6) Construct the weight distribution formula:

[0048] wherein i is the grid number; j represents the described back-propagation jth detection data; m represents the period unit interval between the described current detection, is the weight of the jth detection data.

[0049] In some embodiments, for example, the grid numbered 11, there are 3 historical data, which are respectively spaced 2 period units, 6 period units, and 12 period units. The grid 3 historical data weight can be expressed by the formula:

[0050]

[0051]

[0052] wherein, is the weight of the first historical data of the grid numbered 11, is the weight of the second historical data of the grid numbered 11, is the weight of the third historical data of the grid numbered 11.

[0053] In this embodiment, by inputting the pretreated historical inspection data into the three-dimensional model, and numbering each grid and calculating the time interval, the time difference between each grid and the current inspection data can be more accurately reflected. Based on the updated historical inspection data numbering, the weight distribution formula is constructed and the weight distribution is performed, realizing the auxiliary detection of the local area historical data. By considering the time factor, the processing of the historical data is more timely, and the influence of the data in each grid on the current inspection data can be better evaluated.

[0054] Step 150, based on the target three-dimensional model, generate predicted inspection data, compare and optimize the current inspection data, and obtain target inspection data.

[0055] In some embodiments, the method comprises: establishing a time sequence three-dimensional visualization model of a plurality of local parts based on the target three-dimensional model; generating predicted inspection data based on the time sequence three-dimensional visualization model of the plurality of local parts, comparing and optimizing the current inspection data, and obtaining target inspection data.

[0056] In some embodiments, the time-series three-dimensional visualization model of the plurality of local sites generates predicted inspection data, compares and optimizes the current inspection data to obtain target inspection data, comprising: Rotate the time-series three-dimensional visualization model of the plurality of local sites to the same viewing angle, and intercept a plurality of images; Assign weights to the plurality of images based on a weight assignment formula to generate predicted inspection data; Calculate the mean square error and structural similarity index of the predicted inspection data and the current inspection data, and compare and optimize the current inspection data to obtain target inspection data.

[0057] It is easy to understand that the preprocessed historical data and the generated target three-dimensional model are used to realize time-series three-dimensional observation of the current result, and the historical results of the corresponding sites (meshes) are used to assist in optimizing the current detection. First, a time-series three-dimensional visualization model of local sites is established to facilitate more intuitive comparison and observation, and the specific process is as follows: (1) Observe which meshes are included in the local site, and record the mesh number; (2) For each mesh, determine whether there is historical detection data; (3) If there is historical detection data, generate a historical three-dimensional image of the mesh; (4) Render the historical three-dimensional image of the mesh to display in the same interface; (5) Bind the operation function of the historical three-dimensional images of the mesh, that is, operate (zoom in or out, rotate) any historical three-dimensional model, and other historical three-dimensional models of the same mesh will change at the same rate and in the same proportion, facilitating real-time observation and diagnosis.

[0058] Optionally, the SSIM model can be used to quantitatively evaluate the structural differences of detection results at different times to assist in evaluating the disease progression.

[0059] The time-series three-dimensional visualization model of the plurality of local sites generates predicted inspection data, compares and optimizes the current inspection data to obtain target inspection data, and the specific process is as follows: (1) If there are no less than 3 historical detection results in a certain mesh area of the current detection, first test the SSIM value of the historical detection results (mesh); the detection method is to select 5 viewing angles for the mesh, and intercept two-dimensional images; (2) Calculate the SSIM of the intercepted two-dimensional images and the images of the same viewing angle in the historical detection data; (3) If all the SSIM calculation results are greater than a preset threshold (for example, 0.8), it indicates that the selected images have high similarity (high image consistency, facilitating subsequent image generation), and the historical data similarity of the entire mesh is high, meeting the subsequent image generation requirements; (4) For the current real-time detection, when a certain angle is selected for shooting, the shooting result is recorded as pcurrent, all historical grids in the grid corresponding to the angle are rotated to the same viewing angle, and images are intercepted, which are recorded as p1, p2, … pn, where n represents n historical period observation data; (5) The p1, p2, … pn data are assigned weights to generate a predicted image pfake; (6) The SSIM and MSE of the pcurrent and pfake images are calculated, which can quantitatively evaluate the difference between the predicted result and the true result of the organ region corresponding to the image, and assist in detection.

[0060] In this embodiment, by rotating the time series three-dimensional visualization models of multiple local parts to the same viewing angle and intercepting multiple images, and weighting the multiple images based on a weight distribution formula, more accurate predicted examination data can be generated. By calculating the mean square error and structural similarity index of the predicted examination data and the current examination data, the current examination data is compared and optimized to obtain the target examination data, improving the accuracy and efficiency of comparison and optimization of the examination data, and improving the accuracy and authenticity of the examination data.

[0061] In this embodiment, by establishing time series three-dimensional visualization models of multiple local parts and generating predicted examination data based on these models, efficient comparison and optimization of current examination data can be achieved, and real-time three-dimensional comparison observation is realized by combining the binding of historical three-dimensional models and current examination data, which facilitates efficient auxiliary decision-making of current examination data.

[0062] According to the data optimization method based on multi-source heterogeneous time series assistance provided in the embodiments of the present application, by obtaining the current examination data and historical examination data of the target user and electronically processing the paper image data into electronic image data, the homogeneous data and multi-source heterogeneous data can be effectively integrated, and the quality and consistency of the data are improved. The three-dimensional model is established based on the target organ corresponding to the current examination data, and the model is divided into grids and weighted. By inputting the historical data into the three-dimensional model to generate predicted examination data and comparing and optimizing the predicted examination data with the current examination data, the accuracy and authenticity of the examination data are improved.

[0063] The data optimization method based on multi-source heterogeneous time series assistance provided in the embodiments of the present application can be executed by a data optimization device based on multi-source heterogeneous time series assistance. In the embodiments of the present application, the data optimization device based on multi-source heterogeneous time series assistance is taken as an example to illustrate the data optimization device based on multi-source heterogeneous time series assistance provided in the embodiments of the present application.

[0064] The embodiments of the present application also provide a data optimization device based on multi-source heterogeneous time series assistance, which comprises:Figure 2 As shown in the figure, the data optimization device based on multi-source heterogeneous time sequence assistance includes an acquisition module 210, a first processing module 220, a second processing module 230, a third processing module 240, and an optimization module 250.

[0065] The acquisition module 210 is configured to acquire current examination data and historical examination data of a target user, wherein the historical examination data includes electronic image data and paper image data, and the paper image data is converted into electronic image data through image electronic processing. The first processing module 220 is configured to divide all the electronic image data into homogeneous data and multi-source heterogeneous data, and pre-process the homogeneous data and the multi-source heterogeneous data respectively to obtain pre-processed historical examination data. The second processing module 230 is configured to establish a three-dimensional model based on a target organ corresponding to the current examination data and perform grid division to obtain a three-dimensional model including a plurality of grids. The third processing module 240 is configured to input the pre-processed historical examination data into the three-dimensional model, and perform weight distribution on each grid in the three-dimensional model to obtain a target three-dimensional model. The optimization module 250 is configured to generate predicted examination data based on the target three-dimensional model, compare and optimize the current examination data, and obtain target examination data.

[0066] According to the data optimization method based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application, by acquiring current examination data and historical examination data of a target user and converting paper image data into electronic image data, the homogeneous data and the multi-source heterogeneous data can be effectively integrated, and the quality and consistency of the data are improved. The three-dimensional model is established based on a target organ corresponding to the current examination data, and the model is subjected to grid division and weight distribution. The historical data is input into the three-dimensional model to generate predicted examination data, and the predicted examination data is compared and optimized with the current examination data, thereby improving the accuracy and authenticity of the examination data.

[0067] The data optimization device based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application can realize each process of the data optimization method based on multi-source heterogeneous time sequence assistance Figure 1 The data optimization device based on multi-source heterogeneous time sequence assistance provided in the embodiments of the present application can realize each process of the data optimization method based on multi-source heterogeneous time sequence assistance

[0068] In some embodiments, as shown in Figure 3 The embodiments of the present application also provide an electronic device 300, which includes a processor 301, a memory 302, and a computer program stored in the memory 302 and capable of running on the processor 301. When the program is executed by the processor 301, each process of the above-mentioned data optimization method based on multi-source heterogeneous time sequence assistance is realized, and the same technical effects can be achieved. To avoid repetition, details are not repeated here.

[0069] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0070] The embodiments of the present application further provide a non-transitory computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each process of the data optimization method based on multi-source heterogeneous timing assistance and achieve the same technical effects. To avoid repetition, details are not described herein.

[0071] The processor is the processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0072] The embodiments of the present application further provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the data optimization method based on multi-source heterogeneous timing assistance.

[0073] The processor is the processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0074] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or an instruction to implement each process of the data optimization method based on multi-source heterogeneous timing assistance and achieve the same technical effects. To avoid repetition, details are not described herein.

[0075] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a device-level chip, a device chip, a chip device or a system-on-chip device, etc.

[0076] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or "includes a", does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Additionally, it should be noted that the scope of the methods and apparatus of the present embodiments are not limited by the order of the steps or the sequences of the steps, as some steps can occur in different orders and / or concurrently with other steps besides those depicted and / or discussed herein. Also, described features can be combined in

[0077] From the above description of the embodiments, it is apparent that the above-mentioned method can be realized by means of software and necessary universal hardware platforms, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the multi-source heterogeneous time sequence auxiliary-based data optimization method of various embodiments of the present application.

[0078] In the description of the present application, "first feature" and "second feature" can include one or more of the features.

[0079] In the description of the present application, "a plurality of" means two or more.

[0080] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-described specific embodiments, which are merely illustrative rather than restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

[0081] In the description of the application, reference has been made to descriptive terms such as "one embodiment", "some embodiments", "an embodiment", "example", "specific example" or "some examples" etc. It is emphasized that each of these terms refers to a specific feature, structure, material or characteristic described in connection with a particular embodiment or example. The descriptive terms are not necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0082] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since the scope of the application is defined with respect to the appended claims.

Claims

1. A data optimization method based on multi-source heterogeneous time series assistance, characterized in that: The method comprises: Acquire the target user's current inspection data and historical inspection data, wherein the historical inspection data includes electronic image data and paper image data, and convert the paper image data into electronic image data through electronic image processing; All electronic image data are divided into homogeneous data and multi-source heterogeneous data, and the homogeneous data and multi-source heterogeneous data are preprocessed respectively to obtain preprocessed historical inspection data; Establishing a three-dimensional model of the target organ corresponding to the current examination data and performing mesh division to obtain a three-dimensional model including multiple meshes; The pre-processed historical inspection data is input into the 3D model, and weights are assigned to each grid in the 3D model to obtain the target 3D model; Generate predicted inspection data based on the target 3D model, compare and optimize the current inspection data to obtain the target inspection data.

2. The data optimization method based on multi-source heterogeneous time series assistance according to claim 1 is characterized in that: The electronic processing of paper image data into electronic image data includes: Determine whether the electronic image data and the paper image data in the historical inspection data are duplicated; if the electronic image data and the paper image data are duplicated, delete the paper image data that is duplicated with the electronic image data; if the electronic image data and the paper image data are not duplicated, retain both the electronic image data and the paper image data; Scan all paper image data to obtain electronic image data in png format; After cropping each PNG format electronic image data, a signal-to-noise ratio is calculated to determine whether the signal-to-noise ratio is lower than a preset threshold, the electronic image data with a signal-to-noise ratio lower than the preset threshold is filtered, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained; If the signal-to-noise ratio of the filtered electronic image data is lower than a preset threshold, the electronic image data is discarded, and the electronic image data with a signal-to-noise ratio higher than the preset threshold is retained.

3. The data optimization method based on multi-source heterogeneous time series assistance according to claim 1 is characterized in that: All electronic image data are divided into homogeneous data and multi-source heterogeneous data, and the homogeneous data and multi-source heterogeneous data are preprocessed respectively to obtain preprocessed historical inspection data, including: Compare all electronic image data with the parameters of the current examination data, select electronic image data with the same sensor type and resolution as the current examination data as the homologous data, and number the homologous data. The number is 10, of which the first 8 digits are the image acquisition time and the last 2 digits are the block number corresponding to the 3D organ represented by the image, to obtain the preprocessed homologous data; Treat the electronic image data other than the homologous data as multi-source heterogeneous data, and determine whether the multi-source heterogeneous data has the same type and resolution as the current inspection data; If the multi-source heterogeneous data is of a different type from the current inspection data, convert the multi-source heterogeneous data into data of the same type as the current inspection data; If the resolution of the multi-source heterogeneous data is different from that of the current inspection data, the multi-source heterogeneous data is converted into data with the same resolution as the current inspection data; The multi-source heterogeneous data are numbered with a number of 10, where the first 8 digits are the image acquisition time and the last 2 digits are the block number corresponding to the three-dimensional organ represented by the image, thus obtaining the preprocessed multi-source heterogeneous data.

4. The data optimization method based on multi-source heterogeneous time series assistance according to claim 1 is characterized in that: The step of establishing a three-dimensional model of the target organ corresponding to the current examination data and performing grid division to obtain a three-dimensional model including multiple grids includes: Construct a three-dimensional model of the target organ corresponding to the current examination data; Performing grid processing on the three-dimensional model to obtain multiple grids; Each grid is numbered, and based on the relationship between the last two digits of the block number of the preprocessed historical inspection data and the grid number, the preprocessed homologous data and multi-source heterogeneous data are pasted onto the three-dimensional model surface corresponding to the grid in the form of surface textures to obtain a three-dimensional model including multiple grids.

5. The data optimization method based on multi-source heterogeneous time series assistance according to claim 1 is characterized in that: The pre-processed historical inspection data is input into the three-dimensional model, and weight distribution is performed on each grid in the three-dimensional model to obtain the target three-dimensional model, including: Input the pre-processed historical inspection data into the three-dimensional model and number the historical inspection data of each grid; Calculate the time interval between each grid's historical inspection data and the current inspection data; updating the historical inspection data number of each grid based on the time interval; A weight distribution formula is constructed based on the updated historical inspection data number, and weight distribution is performed on each grid in the three-dimensional model to obtain the target three-dimensional model.

6. The data optimization method based on multi-source heterogeneous time series assistance according to claim 5 is characterized in that: The method of generating predicted inspection data based on the target three-dimensional model and performing comparison and optimization on the current inspection data to obtain target inspection data includes: Establishing a temporal 3D visualization model of multiple local parts based on the target 3D model; Generate predicted inspection data based on the time-series 3D visualization model of multiple local parts, compare and optimize the current inspection data to obtain target inspection data.

7. The data optimization method based on multi-source heterogeneous time series assistance according to claim 6 is characterized in that: The method of generating predicted inspection data based on the time-series three-dimensional visualization model of multiple local parts and performing comparison and optimization on the current inspection data to obtain target inspection data includes: The temporal three-dimensional visualization models of multiple local parts are rotated to the same viewing angle and multiple images are captured; assigning weights to the plurality of images based on a weight assignment formula to generate predicted inspection data; The mean square error and structural similarity index between the predicted inspection data and the current inspection data are calculated, and the current inspection data is compared and optimized to obtain the target inspection data.

8. A data optimization device based on multi-source heterogeneous time series assistance, implemented by the data optimization method based on multi-source heterogeneous time series assistance according to any one of claims 1 to 7, characterized in that: The device comprises: An acquisition module is used to acquire the current inspection data and historical inspection data of the target user, wherein the historical inspection data includes electronic image data and paper image data, and convert the paper image data into electronic image data through electronic image processing; The first processing module is used to divide all electronic image data into homogeneous data and multi-source heterogeneous data, and pre-process the homogeneous data and the multi-source heterogeneous data respectively to obtain pre-processed historical inspection data; A second processing module is used to establish a three-dimensional model based on the target organ corresponding to the current examination data and perform grid division to obtain a three-dimensional model including multiple grids; The third processing module is used to input the pre-processed historical inspection data into the three-dimensional model, and assign weights to each grid in the three-dimensional model to obtain a target three-dimensional model; The optimization module is used to generate predicted inspection data based on the target three-dimensional model, compare and optimize the current inspection data, and obtain the target inspection data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data optimization method based on multi-source heterogeneous timing assistance as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data optimization method based on multi-source heterogeneous timing assistance as described in any one of claims 1 to 7 is implemented.

Images