A method and system for analyzing physical examination images
By working collaboratively between the physical examination terminal and the testing center server, and utilizing lightweight and large-scale deep learning models, combined with automatic pose and angle acquisition and hierarchical caching, the computational resource and storage problems in physical examination image analysis are solved, improving image consistency and fusion clarity, and achieving efficient image analysis and diagnostic assistance.
Patent Information
- Application Number
- CN202611353310.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-17
AI Technical Summary
Existing physical examination image analysis technologies suffer from problems such as high computational resource requirements, large network transmission latency, poor image consistency, high storage costs, and low image fusion contrast.
By deploying a lightweight model on the physical examination terminal to perform basic processing tasks, and deploying a large-scale deep learning model on the testing center server to perform in-depth comparative analysis, the ultrasound image acquisition is automatically triggered by combining the probe posture and angle, and hierarchical caching and iterative convolution fusion technology are adopted.
It enables efficient allocation of computing tasks, improves image consistency and fusion clarity, reduces storage costs and network pressure, and assists in rapid diagnosis.
Smart Images

Figure CN123118554A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image analysis technology, specifically to a method and system for analyzing physical examination images. Background Technology
[0002] With the increasing popularity of health checkups, health checkup centers need to process a large amount of imaging data every day. In routine checkups, ultrasound is widely used due to its non-invasive, real-time, and low-cost nature, while computed tomography (CT) provides clearer anatomical information. Comparing and analyzing images from both modalities helps improve lesion detection rates and diagnostic accuracy. However, current health checkup image analysis technologies still have the following shortcomings: First, multimodal image analysis tasks typically require high computing resources. If all processing is done on local terminals such as ultrasound equipment, the hardware requirements for the terminals are high and the costs are high. If all processing is done on the cloud or a central server, it will result in significant network transmission latency and bandwidth pressure, and the real-time processing capabilities of the terminal devices cannot be fully utilized.
[0003] Secondly, during ultrasound image acquisition, the operator needs to adjust the probe to different positions for different examination sites, such as the abdomen, chest, and limbs. Current acquisition methods mainly rely on manual judgment and lack quantifiable triggering criteria, resulting in poor image consistency between different operators and between multiple acquisitions by the same operator, affecting the reliability of subsequent comparative analysis.
[0004] Third, the storage and retrieval mechanisms for historical image data are unreasonable. Existing technologies often adopt a full storage and full retrieval approach, resulting in high storage costs and slow retrieval speeds; or they only retrieve data randomly on demand, which easily leads to the omission of key comparison information and fails to take into account the different access characteristics of recent high-frequency data and long-term reference data.
[0005] Fourth, traditional methods for fusion and comparison of ultrasound and CT images are relatively simple, usually involving only simple image overlay or feature stitching. They fail to fully utilize dynamic contour information in ultrasound images, such as the minute movements of organ boundaries with breathing and heartbeat, resulting in low contrast and blurred key structures in the fused composite image, which affects the accuracy of subsequent model analysis.
[0006] Therefore, there is an urgent need for a method and system for physical examination image analysis that can reasonably allocate computational tasks, standardize ultrasound acquisition, efficiently store historical data, and deeply integrate multimodal images. Summary of the Invention
[0007] The present invention aims to solve at least one of the technical problems in the prior art, and to this end, provides a method and system for analyzing physical examination images.
[0008] One aspect of the present invention provides a method for analyzing physical examination images, applied to a physical examination terminal, comprising: A multimodal image analysis task is acquired, and the multimodal image analysis task is divided into a basic processing task and a depth contrast analysis task. When the ultrasound probe's orientation angle is detected to meet the preset orientation conditions of the examination site, ultrasound image acquisition is triggered. The basic processing tasks are performed on the acquired ultrasound images by a locally deployed terminal large model to obtain local processing results; The local processing results, the in-depth contrast analysis task, and the acquired ultrasound images are correlated and sent to the detection center server.
[0009] Furthermore, the step of triggering ultrasound image acquisition when the orientation angle of the ultrasound probe meets the conditions of the preset examination site includes: Multiple examination sites are preset, and a corresponding ultrasonic probe posture angle threshold range is set for each examination site; Real-time detection of the ultrasonic probe's attitude and angle; When the posture angle is detected to be within the threshold range corresponding to a certain examination site, the acquisition of ultrasound images of that site is automatically triggered.
[0010] Another aspect of the present invention provides a method for analyzing physical examination images, applied to a testing center server, comprising: The device receives a depth comparison analysis task and associated ultrasound images uploaded by a physical examination terminal, wherein the depth comparison analysis task is obtained by the physical examination terminal after segmenting the multimodal image analysis task. The historical image data of the examinee is retrieved and cached hierarchically according to the time dimension. The image data within the preset period is fully cached, and the image data outside the preset period is only cached as key frames. The computed tomography image corresponding to the ultrasound image is extracted from the historical image data as a reference image. The received ultrasound images are subjected to keyframe filtering to obtain filtered ultrasound images, and the dynamic contour features of the filtered ultrasound images are extracted. Based on the dynamic contour features, the filtered ultrasound image and the reference image are iteratively convolved and fused to generate a composite image; The integrated image is input into the integrated model for analysis, and the analysis results are generated and output.
[0011] Another aspect of the present invention provides a physical examination image analysis system, comprising a physical examination terminal and a testing center server; wherein, The physical examination terminal includes: The task segmentation module is used to acquire multimodal image analysis tasks and segment the multimodal image analysis tasks into basic processing tasks and depth contrast analysis tasks. The attitude acquisition module is used to trigger the acquisition of ultrasound images when the attitude angle of the ultrasound probe meets the preset attitude conditions of the examination site. The local processing module is used to perform the basic processing task on the acquired ultrasound images through a locally deployed terminal large model to obtain local processing results, wherein the local processing results include the basic processed ultrasound images and location annotation information. The data upload module is used to associate the local processing results with the deep comparison analysis task and send them to the detection center server; The testing center server includes: The data receiving module is used to receive the depth comparison analysis task and related ultrasound images uploaded by the physical examination terminal. The local processing results include the ultrasound images after basic processing and the location annotation information. The historical data processing module is used to retrieve the historical image data of the examinee. The historical image data is cached hierarchically according to the time dimension. The image data within the preset period is fully cached, and the image data exceeding the preset period is only cached as key frames. The computed tomography image corresponding to the ultrasound image is extracted from the historical image data as a reference image. An ultrasound image processing module is used to perform keyframe filtering on the ultrasound images in the local processing results to obtain filtered ultrasound images. The comparison model processing module is used to extract the dynamic contour features of the screened ultrasound images, and based on the dynamic contour features, iteratively convolutionally fuse the screened ultrasound images and the reference images to generate a comprehensive image. The integrated model analysis module is used to input the integrated image into the integrated model for analysis, and generate and output the analysis results.
[0012] This invention provides a method and system for analyzing physical examination images. Basic processing tasks are performed by a lightweight model on the physical examination terminal, while in-depth analysis tasks are handled by a large model on the testing center server, balancing real-time performance and computational power requirements. Ultrasound acquisition is automatically triggered by probe posture and angle thresholds, improving the consistency of images from different operators and at different times. Historical images are cached in a time-based hierarchy, with recent images fully cached and older keyframes cached, balancing storage efficiency and retrieval speed. Iterative convolutional fusion based on dynamic contour coefficients highlights dynamic ultrasound edges, significantly improving the clarity after fusion with computed tomography images. Combining medical record tags and abnormal distribution statistics, the system intuitively outputs the proportion of abnormal areas, aiding in rapid diagnosis. Attached Figure Description
[0013] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a method for analyzing physical examination images applied to a physical examination terminal, provided in one embodiment of this application. Figure 2 This is a flowchart illustrating a method for analyzing physical examination images applied to a testing center server, provided in one embodiment of this application. Figure 3 This is a schematic diagram of the structure of a physical examination image analysis system provided in one embodiment of this application. Detailed Implementation
[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0015] I. Meaning of Key Technical Terms To facilitate understanding of this invention, the meanings of key technical terms involved in this application are explained.
[0016] Multimodal image analysis task: refers to the complete analysis process that requires processing medical images of different modalities such as ultrasound, CT, MRI (Magnetic Resonance Imaging), X-ray, etc., simultaneously or sequentially, and performing cross-comparison, fusion, feature extraction and anomaly detection.
[0017] Basic processing tasks refer to preprocessing operations with relatively small computational load and high real-time requirements, such as image denoising, normalization, size unification, and basic filtering. These tasks are suitable for execution by a lightweight terminal with a large model on a resource-constrained physical examination terminal.
[0018] Deep contrast analysis tasks refer to analysis tasks that require a large amount of computing resources and involve complex models such as deep convolutional networks, iterative fusion, and multimodal contrast. Data needs to be uploaded to the detection center server and completed by a high-performance multimodal large model.
[0019] Terminal large model: A lightweight deep learning model, such as MobileNet, ShuffleNet or MobileViT, deployed on physical examination terminal devices to perform basic processing tasks. It features small number of parameters, fast inference speed and low power consumption.
[0020] Dynamic contour coefficient: used to characterize the intensity of each pixel in an ultrasound image belonging to a dynamic contour (such as the edge of an organ boundary moving with respiration). The value ranges from 0 to 1 and is obtained by normalizing the gradient value extracted by the edge detection algorithm.
[0021] Multimodal large model: Deployed on the server of the detection center, it is a large-scale deep learning model that can simultaneously process and fuse medical image data of multiple modalities, including multiple sub-models such as comparison model and comprehensive model.
[0022] II. Method Implementation Example 1 This embodiment details a method for analyzing physical examination images, applied to a physical examination terminal. For example... Figure 1 As shown, the method mainly includes the following steps.
[0023] Step S101: Obtain the multimodal image analysis task and divide the multimodal image analysis task into a basic processing task and a depth contrast analysis task.
[0024] This step aims to address the inefficiency caused by all tasks piling up on the terminal or all data being uploaded to the server in existing technologies. The medical examination terminal, such as a portable ultrasound diagnostic instrument, first receives or generates a multimodal image analysis task. This task may include sub-tasks such as real-time enhancement and denoising of the currently acquired ultrasound images, and comparative analysis with the examinee's historical CT images.
[0025] The system dynamically divides tasks into two parts based on computational complexity, real-time requirements, and terminal resource availability: basic processing tasks and deep contrast analysis tasks. For example, basic processing tasks include image normalization, denoising, and size unification, while deep contrast analysis tasks include iterative convolutional fusion of multimodal images and comprehensive model inference. The segmentation strategy is determined by preset rules; for instance, tasks involving deep inference from convolutional neural networks are classified as deep contrast analysis tasks, while simple pixel-level transformations are classified as basic processing tasks.
[0026] Step S102: When the posture angle of the ultrasound probe is detected to meet the preset posture conditions of the examination site, the acquisition of ultrasound images is triggered.
[0027] This step aims to automate and standardize the triggering of ultrasound image acquisition. Traditional ultrasound examinations require the operator to manually press the acquisition button, and different operators may have different interpretations of the probe angle for the same area. This embodiment uses an angle sensor integrated into the medical examination terminal to detect the attitude angle of the ultrasound probe relative to a reference plane, such as a horizontal plane, in real time.
[0028] Specifically, the system pre-stores threshold ranges for probe posture angles for various examination sites, such as the abdomen, chest, neck, and limbs. For example, for abdominal examinations, the probe angle with the horizontal plane can be set to 45°±5°; for chest examinations, it can be set to 0°±5°; for limb long axis examinations, it can be set to 60°±5°; and for perineal examinations, it can be set to 90°±5°, etc. These threshold ranges can be obtained through clinical statistics or user-defined settings.
[0029] The medical examination terminal continuously reads the real-time posture angle of the probe and compares it with the threshold range of each body part. When it detects that the current posture angle falls within the threshold range corresponding to a certain examination part, it automatically triggers the acquisition of ultrasound images without manual intervention from the operator. The acquired ultrasound images will include annotation information for that part, such as using pixel coordinate boxes, for example, the annotation box for the abdomen is set to pixel coordinates (220, 160) to (780, 590), for subsequent processing.
[0030] Step S103: Perform the basic processing task on the acquired ultrasound images through the locally deployed terminal large model to obtain local processing results, wherein the local processing results include the basic processed ultrasound images and location annotation information.
[0031] Specifically, the physical examination terminal is pre-deployed with a lightweight terminal model, such as the MobileViT-v2 neural network. This model is compressed and quantized, enabling it to run in real time on embedded devices. After ultrasound image acquisition, the terminal model receives the basic processing tasks segmented in step S101, performing operations such as Gaussian filtering for noise reduction, pixel value normalization to 0-255, and scaling images from different sources to a preset pixel size, such as 1024×1024 pixels, to generate structured local processing results. These results include processed image data and intermediate feature maps, used for subsequent uploading or local display.
[0032] Since the basic processing tasks are completed locally, the network latency and bandwidth consumption caused by uploading all the original massive image data are avoided. At the same time, the low latency requirements for some real-time feedback, such as displaying enhanced ultrasound images on the local screen, are also guaranteed.
[0033] Step S104: Associate the local processing result with the deep comparison analysis task and send it to the detection center server.
[0034] The physical examination terminal associates and packages the local processing results obtained in step S103 with the depth contrast analysis task segmented in step S101, and uploads them to the testing center server. The amount of data uploaded is significantly reduced compared to the original full-volume images, and the depth contrast analysis task is executed on the server side, making full use of the server's computing power.
[0035] III. Method Example 2 This embodiment details a physical examination image analysis method applied to a testing center server, such as... Figure 2 As shown, the method mainly includes the following steps.
[0036] Step S201: Receive the depth comparison analysis task and local processing results uploaded by the physical examination terminal; wherein, the local processing results include the ultrasound images and location annotation information after basic processing.
[0037] The testing center server receives data packets from the physical examination terminal through its communication interface and parses out the depth contrast analysis task and the local processing results. The depth contrast analysis task is obtained by segmenting the multimodal image analysis task by the physical examination terminal; the local processing results include the basic processed ultrasound images and site annotation information, such as the ultrasound image sequence after denoising, normalization and size unification by the terminal's large model, and the annotation boxes in the form of pixel coordinates.
[0038] Step S202: Retrieve the historical image data of the examinee. The historical image data is cached hierarchically according to the time dimension. The image data within the preset period is cached in full, and the image data exceeding the preset period is cached only for key frames. Extract the computed tomography image of the corresponding part of the ultrasound image from the historical image data as a reference image.
[0039] This step addresses the technical challenges of efficient storage and retrieval of historical image data. The testing center's server employs a tiered caching strategy: for each examinee, all image data from the most recent preset period (e.g., one year), including CT, MRI, and ultrasound images, are fully cached and stored on high-performance storage media such as solid-state drives (SSDs); for historical images older than this period, only keyframes are cached, such as representative slices of each sequence or abnormal frames annotated by doctors, stored on low-cost, high-capacity storage media such as hard disk drives (HDDs). Keyframe selection can utilize content-based techniques, such as selecting frames containing the most complete anatomical structures and the highest image quality.
[0040] When the server needs to retrieve reference images, it first determines the target area based on the location annotation information in the ultrasound images, and then searches for the corresponding CT images of the patient's area in the hierarchical cache. If full data is available recently, the complete CT sequence is retrieved directly; if only long-term keyframes are available, the keyframes are retrieved. This mechanism ensures retrieval speed while significantly saving storage space.
[0041] Step S203: Perform keyframe filtering on the ultrasound images in the local processing results to obtain filtered ultrasound images.
[0042] The locally processed results uploaded by the medical examination terminal include basic processed ultrasound images, typically a video stream, such as a 10-second, 25 frames per second video stream, which contains a large amount of redundant information and motion artifacts. Keyframe filtering aims to select frames with stable quality and relatively fixed location. Specifically, based on the location annotation information, the system calculates the inter-frame difference of the ultrasound images: comparing the pixel differences between two adjacent frames within the annotation box area, when the inter-frame pixel deviation is less than a preset threshold, such as 5%, it is determined to be a stable frame and selected as a keyframe. After filtering, a set of keyframes with relatively fixed positions is obtained, and the pixel coordinate deviation between adjacent keyframes is usually no more than ±10 pixels. Then, a representative frame is selected from these keyframes, such as the frame with the highest image quality score, or the frame with the smallest pixel deviation within the annotation box, as the filtered ultrasound image for subsequent processing.
[0043] Step S204: Extract the dynamic contour features of the filtered ultrasound image, and based on the dynamic contour features, perform iterative convolution fusion on the filtered ultrasound image and the reference image to generate a comprehensive image.
[0044] Specifically, it includes the following steps: Step S2041: Calculate the dynamic contour coefficient of each pixel in the filtered ultrasound image, and form a dynamic contour image based on the dynamic contour coefficient of each pixel.
[0045] Specifically, firstly, the dynamic contour features of the filtered ultrasound images are extracted. The dynamic contour features reflect the edge gradient changes of the organ boundaries in the ultrasound images caused by physiological activities such as breathing and heartbeat. For example, edge detection algorithms such as the Canny edge detection algorithm are used to extract the contour pixel gradient value of each pixel in the filtered ultrasound images. Then, the dynamic profile coefficient is calculated using the following formula: in, The two-dimensional coordinates of pixels in the filtered ultrasound images. For pixels The contour pixel gradient value, For pixels The dynamic contour coefficient ranges from 0 to 1, with a higher value indicating a higher probability that the pixel belongs to the dynamic contour. The dynamic contour coefficients of all pixels constitute a dynamic contour image of the same size as the filtered ultrasound image.
[0046] Step S2042: Input the dynamic contour image and the reference image into the contrast model. In the convolutional layer of the contrast model, perform a convolution operation with the dynamic contour coefficients as weights to obtain the output tensor of each convolutional layer.
[0047] Specifically, the contrast model is a pre-trained convolutional neural network. For example, the contrast model contains four convolutional layers with kernel sizes of 3×3 and channel numbers of 16, 32, 64, and 128, respectively. The dynamic contour image and the reference image extracted in step S202 are used as input together; they can be concatenated or input separately. In each convolutional layer, the convolution operation incorporates dynamic contour coefficients as weights, and the convolution operation is performed using the following formula: in, This is the output feature map of the c-th channel of the nth convolutional layer. The input tensor is used for the previous convolutional layer; for the first layer, it is the original input. Let K be the input tensor for the convolution kernel, and K be the set of convolution kernels. For dynamic profile coefficients, Here, f is the bias term, and f() is the activation function such as ReLU. By... When multiplied into the convolution result, the model will pay more attention to regions with obvious dynamic contours during training and inference, thereby improving the contrast of key structures in the fused image.
[0048] Step S2043: Multiply the dynamic contour image with the output tensors of each convolutional layer at the pixel level, and weight the output tensors using global dynamic contour coefficients to obtain the comparison output tensor.
[0049] Specifically, the dynamic contour image is multiplied pixel-wise with the output tensors of each convolutional layer using the following formula: in, For dynamic contour images at pixel points pixel value at The product tensor corresponding to the nth convolutional layer at the pixel point Pixel value at; This operation further enhances the response in the contour region and suppresses noise in the non-contour region; The comparison output tensor is obtained by weighting using global dynamic contour coefficients using the following formula: in, This refers to the global dynamic contour coefficient, which is the sum of the dynamic contour coefficients of all pixels. To compare the output tensor at the pixel level Pixel value at; The global dynamic contour coefficient reflects the overall contour richness of the current ultrasound image. Using it as a scaling factor, the fusion intensity can be adaptively adjusted according to the clarity of the image content.
[0050] Step S2044: The comparison output tensor and the dynamic contour image are weighted and superimposed to generate an enhanced image.
[0051] The enhanced image is generated using the following weighted stacking formula: in, To enhance the image at the pixel level Pixel value at that location, These are fixed weighting coefficients for dynamic contour images; The weighted superimposed enhanced image retains the basic information of the original contour while incorporating features enhanced by convolution and global weighting.
[0052] Step S2045: The enhanced image is used as a new dynamic contour image, and the above steps are executed iteratively until the preset number of iterations is met.
[0053] The enhanced image generated in step S2044 is used as a new dynamic contour image, and steps S2042 to S2044 are repeated until the preset number of iterations is reached. Iteration gradually amplifies the effective features in multimodal fusion, making the final image structure clearer. It should be noted that the dynamic contour coefficient α(x,y) remains unchanged during the iteration process to avoid introducing new noise.
[0054] Step S2046: The enhanced image after reaching the required number of iterations is weighted and superimposed with the reference image to generate the composite image.
[0055] After iteration, the final enhanced image after reaching the preset number of iterations is superimposed with the reference image from step S202 using a weighted superposition. The weight of the reference image is a fixed coefficient γ, preferably 0.7, and the weight of the enhanced image is a dynamic contour coefficient α(x,y). The enhanced image after the preset number of iterations is used as the final enhanced image, and a composite image is generated by superimposing it with the reference image using the following weighted superposition formula: in, To synthesize images at the pixel level Pixel value at that location, For reference image at pixels Pixel value at that location, Fixed weighting coefficients for the reference image. To ultimately enhance the image at the pixel level The pixel value at that location is determined by the enhanced image at the pixel point. pixel value at Obtained after iterating to a preset number of iterations; The generated U is the composite image, which combines the high-resolution anatomical advantages of CT images with the dynamic contour features of ultrasound images, providing high-quality input for subsequent analysis.
[0056] Step S205: Input the integrated image into the integrated model for analysis, generate and output the analysis results.
[0057] Specifically, the integrated model is a deep neural network deployed on the detection center server, such as a multimodal fusion network based on Swin-Transformer, or a detection network for specific organs. This model receives the integrated image generated in step S204 and performs analytical tasks such as anomaly detection, lesion segmentation, and benign / malignant classification.
[0058] This step also includes the tagging and statistical processing of the examinee's medical records. Specifically, the system acquires the examinee's electronic medical records, including past medical history, laboratory indicators, and clinical symptoms. Based on the analysis results of comprehensive imaging, standardized analysis tags such as "fatty liver - mild" and "kidney stones - single" are generated using natural language processing and rule matching methods. Then, the percentage of abnormalities in each abnormal location is calculated using the following formula: in, This represents the percentage of abnormalities in abnormal locations. The weight is the weight of the i-th abnormal site. This weight is preset according to clinical importance, such as the weight of the liver being 0.2, the weight of the gallbladder being 0.1, the weight of the kidney being 0.15, and the weight of the heart being 0.25, etc. is the weighted number of the i-th abnormal part; m is the total number of abnormal parts. The weight is the weight of the j-th examination, such as 0.6 for CT examination and 0.4 for ultrasound examination; This represents the number of abnormal parts corresponding to the j-th inspection; Let be the total number of abnormalities in the j-th examination; p be the total number of examination types; and T be the total number of multimodal examinations. The final output of the abnormality distribution statistics can be presented in tabular or graphical form to help doctors quickly understand the abnormality distribution trend of the physical examination population.
[0059] IV. System Implementation Examples See Figure 3 Another embodiment of the present invention provides a physical examination image analysis system 300, including a physical examination terminal 310 and a testing center server 320. This system is capable of performing the corresponding steps in the above method embodiments.
[0060] The physical examination terminal 310 includes: The task segmentation module 311 is used to acquire a multimodal image analysis task and segment the multimodal image analysis task into a basic processing task and a depth contrast analysis task. The attitude acquisition module 312 is used to trigger the acquisition of ultrasound images when the attitude angle of the ultrasound probe meets the preset attitude conditions of the examination site. The local processing module 313 is used to perform the basic processing task on the acquired ultrasound images through the locally deployed terminal large model to obtain the local processing result, wherein the local processing result includes the ultrasound images after basic processing and the location annotation information. Data upload module 314 is used to associate the local processing results with the deep comparison analysis task and send them to the detection center server; The testing center server 320 includes: The data receiving module 321 is used to receive the depth comparison analysis task and related local processing results uploaded by the physical examination terminal, wherein the local processing results include the ultrasound images and site annotation information after basic processing. The historical data processing module 322 is used to retrieve the historical image data of the examinee. The historical image data is cached hierarchically according to the time dimension. The image data within the preset period is fully cached, and the image data exceeding the preset period is only cached as key frames. The computed tomography image of the corresponding part of the ultrasound image is extracted from the historical image data as a reference image. The ultrasound image processing module 323 is used to perform keyframe filtering on the ultrasound images in the local processing results to obtain filtered ultrasound images. The comparison model processing module 324 is used to extract the dynamic contour features of the screened ultrasound image, and based on the dynamic contour features, iteratively convolutionally fuse the screened ultrasound image and the reference image to generate a comprehensive image. The integrated model analysis module 325 is used to input the integrated image into the integrated model for analysis, and generate and output the analysis results.
[0061] It should be noted that the system provided in this embodiment corresponds to the technical solutions that can be used to execute the various method embodiments. Its implementation principle and technical effect are similar to the methods, and will not be repeated here.
[0062] V. Specific Application Examples To make the technical solution and beneficial effects of the present invention clearer, the present invention will be described in detail below with reference to a complete example of its application in the annual physical examination work of a community health service center. This example is not intended to limit the present invention, but is merely a specific application scenario.
[0063] This example is applied to the annual physical examination work of a community health service center. The center serves 5,000 permanent residents in its jurisdiction, and the examination items include abdominal ultrasound and abdominal CT multimodal imaging examinations. It is necessary to achieve rapid image transmission, analysis, and historical data comparison. This example uses the statistics of one examinee, A, and 50 examinees on the same day as an illustration.
[0064] 1. Configuration and operation of the physical examination terminal: A portable ultrasound diagnostic instrument was used as the physical examination terminal, with a built-in MobileViT-v2 lightweight terminal model. The attitude acquisition module integrates an angle sensor, and the ultrasound probe is a convex array probe with a frequency of 3.5MHz. Pre-set probe attitude angle trigger thresholds for each body part are as follows: abdomen 45°±5°, chest 0°±5°, limbs 60°±5°, and perineum 90°±5°. The annotation box uses a pixel coordinate frame, with the abdominal annotation box set to pixel coordinates (220, 160) to (780, 590).
[0065] 2. Ultrasonic Acquisition and Preprocessing: When performing an abdominal ultrasound examination on examinee A, the posture acquisition module detected that the angle between the operator's probe and the horizontal plane was 46°, which was within the abdominal posture trigger threshold of 45°±5°. The system automatically triggered ultrasound image acquisition. The acquired abdominal ultrasound images had a resolution of 1024×1024 pixels, a frame rate of 25fps, an acquisition time of 10s, and a total of 250 frames.
[0066] The MobileViT-v2 model of the physical examination terminal performs basic preprocessing on the acquired ultrasound images, including Gaussian filtering for noise reduction, pixel value normalization to 0-255, and a uniform size of 1024×1024 pixels. After preprocessing, only the depth contrast analysis task and related local processing results are uploaded to the testing center server, with an uploaded data volume of 8.2MB, which is about 90% less than the full image upload.
[0067] 3. Hierarchical caching and retrieval of historical data on the server side: After receiving the data, the testing center server retrieves the historical abdominal imaging data of examinee A. Abdominal images from 2025 are fully cached, and the server directly retrieves the full CT images. Abdominal images from 2023 are only cached as keyframes; in this case, because recent full data is available, there is no need to retrieve older keyframes. This tiered caching mechanism ensures that the retrieval time is only about 0.8 seconds, an efficiency improvement of approximately 85% compared to the traditional full-data retrieval method.
[0068] 4. Screening of keyframes in ultrasound images: The ultrasound image processing module of the testing center's server performed keyframe screening on the 250 uploaded abdominal ultrasound images, using the inter-frame difference method, defining stable frames as those with an inter-frame pixel deviation of less than 5%. After screening, 22 keyframe images with relatively fixed positions were selected, with adjacent keyframe pixel coordinate deviations not exceeding ±10 pixels. These selected keyframes, along with the 1024×1024 pixel abdominal CT image of the examinee A, served as the basis for the comparative model analysis.
[0069] 5. Comparison of iterative convolutional fusion processing in the model: The Canny edge detection algorithm was used to extract the dynamic contour of abdominal ultrasound keyframes, and the dynamic contour coefficient α(x,y) of each pixel was calculated. One frame was selected from the filtered keyframes. The maximum value of the contour pixel gradient value G(x,y) of this frame was 255, and the value of G(x,y) of pixel (300,200) was 128. Therefore, the α(300,200) of this pixel was 128 / 255≈0.502. Dynamic contour images and abdominal CT images were input into a comparison model. The model had four convolutional layers with 3×3 kernels of 16, 32, 64, and 128 kernels respectively. The activation function was ReLU f(x)=max(0,x). Convolution operations were then performed. The bias term of channel 16 in the second convolutional layer. =0.02; Perform pixel-level multiplication operation on a single channel. For pixel (300, 200), D(300, 200) = 255. ,but ; Calculate the sum of the dynamic contours ∑α(x,y)=8965.2, and execute... Pixel (300, 200) ; Set a fixed weight β=0.3 for the dynamic contour image and perform weighted overlay. Pixel (300, 200) Use the enhanced image as a new dynamic contour image and iterate 3 times. Set the fixed weight of CT images γ=0.7, and execute. For pixel (300, 200), CT(300, 200) = 200, then U(300, 200) = 0.7 × 200 + 0.502 × 2.066 × 10^8 ≈ 1.037 × 10^8, generating a multimodal composite image of the abdomen. This composite image clearly shows the fusion enhancement effect between the liver edge and the CT liver parenchyma structure.
[0070] 6. Comprehensive model analysis and statistics: The integrated model (a multimodal fusion network based on Swin-Transformer) analyzes the integrated images of A and outputs the analysis label "no obvious abnormalities were found in the abdominal images".
[0071] The medical record module retrieves Mr. Zhang's medical records (including basic information, past medical history with no abdominal-related history, and normal liver and kidney function test results). The case analysis module confirms the labels based on the comprehensive imaging analysis results.
[0072] The statistics module categorized and analyzed the results of 50 physical examination participants on that day. Among them, 3 people had abdominal abnormalities, with the abnormalities occurring in the liver in 2 cases and the gallbladder in 1 case.
[0073] Using formula Calculate the percentage of abnormal areas and assign a weight to the liver. Gallbladder weight CT scan weight Weight of ultrasound examination Total number of multimodal inspections (50 people undergoing physical examinations × 2 tests), total number of abnormal CT scans Total number of abnormalities detected by ultrasound , , , , .
[0074] Calculation process: The final analysis table of abnormal sites was generated: liver abnormalities accounted for 80%, gallbladder abnormalities accounted for 20%, and this data was synchronized to the physical examination report system. This statistical result provides quantitative data support for community health management.
[0075] As can be seen from the above examples, this invention rationally divides the multimodal image analysis task, using a large terminal model to handle basic tasks and a large server-side multimodal model to handle depth contrast tasks, effectively reducing network transmission pressure and terminal hardware requirements; automatic acquisition triggered by probe posture ensures the standardization of ultrasound images; a hierarchical caching mechanism enables efficient retrieval of historical data; dynamic contour-weighted iterative convolution fusion significantly improves the fusion quality of ultrasound and CT images; and an integrated model and statistical module enable automated and quantitative generation of physical examination reports.
[0076] The above description is merely a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to the specific combination of the above-described technical features, but should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.
Claims
1. A method for analyzing physical examination images, characterized in that, Applications in health checkup terminals include: A multimodal image analysis task is acquired, and the multimodal image analysis task is divided into a basic processing task and a depth contrast analysis task. When the ultrasound probe's orientation angle is detected to meet the preset orientation conditions of the examination site, ultrasound image acquisition is triggered. The basic processing task is performed on the acquired ultrasound images by a locally deployed terminal large model to obtain local processing results, wherein the local processing results include the basic processed ultrasound images and location annotation information. The local processing results are associated with the deep comparison analysis task and sent to the detection center server.
2. The method according to claim 1, characterized in that, The step of triggering ultrasound image acquisition when the ultrasound probe's orientation angle meets the conditions of a preset examination site includes: Several examination sites are preset, and a corresponding ultrasonic probe posture angle threshold range is set for each examination site; Real-time detection of the ultrasonic probe's attitude and angle; When the posture angle is detected to be within the threshold range corresponding to a certain examination site, the acquisition of ultrasound images of that site is automatically triggered.
3. A method for analyzing physical examination images, characterized in that, Applications to the testing center server include: Receive the depth comparison analysis task and related local processing results uploaded by the physical examination terminal, wherein the local processing results include the ultrasound images and site annotation information after basic processing; The historical image data of the examinee is retrieved and cached hierarchically according to the time dimension. The image data within the preset period is fully cached, and the image data outside the preset period is cached only for key frames. The computed tomography image corresponding to the ultrasound image is extracted from the historical image data as a reference image. The ultrasound images in the local processing results are filtered by keyframes to obtain filtered ultrasound images. Extract the dynamic contour features of the filtered ultrasound images, and based on the dynamic contour features, perform iterative convolution fusion on the filtered ultrasound images and the reference images to generate a composite image; The integrated image is input into the integrated model for analysis, and the analysis results are generated and output.
4. The method for analyzing physical examination images according to claim 3, characterized in that, The step of extracting dynamic contour features from the filtered ultrasound images, and then iteratively convolving and fusing the filtered ultrasound images and the reference image based on these dynamic contour features to generate a composite image, includes: Calculate the dynamic contour coefficient of each pixel in the filtered ultrasound image, and form a dynamic contour image based on the dynamic contour coefficient of each pixel. The dynamic contour image and the reference image are input into the comparison model. In the convolutional layer of the comparison model, the dynamic contour coefficients are used as weights for convolution operation to obtain the output tensor of each convolutional layer. The dynamic contour image is multiplied pixel-level by the output tensor of each convolutional layer, and weighted using global dynamic contour coefficients to obtain the contrast output tensor. The contrast output tensor and the dynamic contour image are weighted and superimposed to generate an enhanced image; The enhanced image is used as a new dynamic contour image, and the above steps are performed iteratively until the preset number of iterations is met. The enhanced image after reaching the required number of iterations is weighted and superimposed with the reference image to generate the composite image.
5. The method for analyzing physical examination images according to claim 4, characterized in that, The calculation of the dynamic contour coefficient of each pixel in the filtered ultrasound image, and the formation of a dynamic contour image based on the dynamic contour coefficient of each pixel, includes: An edge detection algorithm is used to extract the contour pixel gradient values of the filtered ultrasound images; The dynamic profile coefficient is calculated using the following formula: in, The two-dimensional coordinates of pixels in the filtered ultrasound images. For pixels The contour pixel gradient value, For pixels The dynamic profile coefficient; The dynamic contour coefficients of all pixels constitute a dynamic contour image of the same size as the filtered ultrasound image.
6. The method for analyzing physical examination images according to claim 5, characterized in that, The iterative convolutional fusion includes: The convolution operation is performed using the following formula: in, This is the output feature map of the c-th channel of the nth convolutional layer. Input tensors to the previous convolutional layer, Let K be the input tensor for the convolution kernel, and K be the set of convolution kernels. For dynamic profile coefficients, Here, f is the bias term, and f() is the activation function; The dynamic contour image is multiplied pixel-wise with the output tensor of each convolutional layer using the following formula: in, For dynamic contour images at pixel points pixel value at The product tensor corresponding to the nth convolutional layer at the pixel point Pixel value at; The comparison output tensor is obtained by weighting using global dynamic contour coefficients using the following formula: in, For global contour coefficients, To compare the output tensor at the pixel level Pixel value at; The enhanced image is generated using the following weighted stacking formula: in, To enhance the image at the pixel level Pixel value at that location, Fixed weighting coefficients for dynamic contour images; The enhanced image after a preset number of iterations is used as the final enhanced image, and a composite image is generated by combining it with the reference image using the following weighted superposition formula: in, To synthesize images at the pixel level Pixel value at that location, For reference image at pixels Pixel value at that location, Fixed weighting coefficients for the reference image. To ultimately enhance the image at the pixel level The pixel value at that location is determined by the enhanced image at the pixel point. pixel value at It is obtained after iterating to the preset number of iterations.
7. The method for analyzing physical examination images according to claim 3, characterized in that, The step of inputting the integrated image into the integrated model for analysis, generating and outputting analysis results includes: Based on the comprehensive images, the medical records of the examinees are tagged to generate analysis tags; The analysis tags are classified and statistically analyzed to output multimodal image anomaly distribution statistics. The anomaly distribution statistics include the percentage of anomalies in each anomaly region, which is calculated using the following formula: in, This represents the percentage of abnormalities in abnormal locations. Let i be the weight of the i-th abnormal part. Let m be the weighted count of the i-th abnormal part, and m be the total number of abnormal part types. The weight of the j-th inspection. Let j be the number of the i-th abnormal parts that are checked. Let p be the total number of anomalies checked in the j-th inspection, p be the total number of inspection types, and T be the total number of multimodal inspections.
8. The method for analyzing physical examination images according to claim 3, characterized in that, The step of performing keyframe filtering on the ultrasound images in the local processing results to obtain filtered ultrasound images includes: Obtain the location annotation information corresponding to the ultrasound image in the local processing result; Based on the location annotation information, the ultrasound image in the local processing result is matched with the reference image corresponding to that location; The frame difference is calculated on the matched ultrasound images. Frames with a pixel deviation of less than a preset threshold are determined to be stable frames, and the selected stable frame images are used as the selected ultrasound images.
9. A physical examination image analysis system, characterized in that, This includes physical examination terminals and testing center servers; among them, The physical examination terminal includes: The task segmentation module is used to acquire multimodal image analysis tasks and segment the multimodal image analysis tasks into basic processing tasks and depth contrast analysis tasks. The attitude acquisition module is used to trigger the acquisition of ultrasound images when the attitude angle of the ultrasound probe meets the preset attitude conditions of the examination site. The local processing module is used to perform the basic processing task on the acquired ultrasound images through a locally deployed terminal large model to obtain local processing results, wherein the local processing results include the basic processed ultrasound images and location annotation information. The data upload module is used to associate the local processing results with the deep comparison analysis task and send them to the detection center server; The testing center server includes: The data receiving module is used to receive the depth comparison analysis task and related local processing results uploaded by the physical examination terminal, wherein the local processing results include the ultrasound images and site annotation information after basic processing. The historical data processing module is used to retrieve the historical image data of the examinee. The historical image data is cached hierarchically according to the time dimension. The image data within the preset period is fully cached, and the image data exceeding the preset period is only cached as key frames. The computed tomography image corresponding to the ultrasound image is extracted from the historical image data as a reference image. An ultrasound image processing module is used to perform keyframe filtering on the ultrasound images in the local processing results to obtain filtered ultrasound images. The comparison model processing module is used to extract the dynamic contour features of the screened ultrasound images, and based on the dynamic contour features, iteratively convolutionally fuse the screened ultrasound images and the reference images to generate a comprehensive image. The integrated model analysis module is used to input the integrated image into the integrated model for analysis, and generate and output the analysis results.
10. A physical examination image analysis system according to claim 9, characterized in that, The comprehensive model analysis module also includes a medical record analysis module and a statistics module: The medical record analysis module is used to tag the examinee's medical record data based on the integrated image and generate analysis tags. The statistical module is used to classify and statistically analyze the analysis labels and output the statistical results of multimodal image anomaly distribution.
