Multi-mode intelligent image system and photographing method thereof

Abstract

The invention relates to the technical field of medical image processing, in particular to a multi-modal intelligent image system and a photographing method thereof, and the system comprises an information collection module which is used for obtaining posture information and an image set of a user; the body surface model building module is used for generating a three-dimensional body surface model of the user; the image fusion module is used for realizing spatial alignment and point-by-point matching of different modal images; the risk area judgment module is used for identifying a potential risk area and a newly added risk area; and the region division module is used for generating a repeated imaging region of a cost period based on the target risk region. According to the method, the three-dimensional body surface model is used as a unified space reference, cross-cycle stable alignment of the multi-modal images is achieved, the accurate target risk area is obtained, the body surface section needing repeated collection is planned according to the target risk area, invalid repeated shooting is reduced, the image collection efficiency is improved, and the image collection efficiency is improved. And meanwhile, the continuity and the reliability of the risk part in cross-cycle imaging are ensured.

Description

A multimodal intelligent imaging system and its photographic method Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a multimodal intelligent imaging system and its imaging method. Background Technology

[0002] With the increasing demand for skin disease screening, follow-up of superficial tumors, and long-term health monitoring, body surface imaging acquisition is gradually evolving from single-modality to multi-modal fusion. However, existing body surface imaging systems still have several limitations: First, single-modality images cannot simultaneously reflect color, texture, and three-dimensional deformation information, leading to insufficient identification of early abnormalities or mildly developing lesions; second, traditional body surface acquisition relies on manual planning of imaging sites, and the judgment of repeated imaging areas is inaccurate, which may miss key areas and easily lead to unnecessary repeated imaging, increasing the examination burden; third, existing systems lack a dynamic risk assessment mechanism based on time-series comparison, making it impossible to quantitatively analyze the occurrence, expansion, or migration trends of body surface abnormalities; at the same time, the image positioning is inconsistent in different acquisition cycles, making it difficult to accurately correspond the images on the three-dimensional model, thus hindering the realization of the value of multi-cycle monitoring.

[0003] Furthermore, existing image processing platforms typically only perform static detection of anomalies, lacking automatic optimization strategies for image blur, artifact interference, or occlusion of sensitive areas. This results in unstable image quality and low efficiency in repeated inspections. Therefore, it is necessary to provide a system that can integrate multimodal images, construct accurate 3D body surface models, automatically identify anomalies, and intelligently plan repeated acquisition areas to improve the accuracy and efficiency of anomaly detection, tracking, and management.

[0004] Chinese Patent Publication No. CN111449665A discloses an intelligent image diagnostic system, including: an intelligent diagnostic workstation, an intelligent gateway device, an X-ray film acquisition module, and a human-computer interaction module. The X-ray film acquisition module and the human-computer interaction module are both connected to the intelligent gateway device, and the intelligent gateway device is connected to the intelligent diagnostic workstation.

[0005] It is evident that existing medical image processing technologies and systems typically only acquire and transmit data for a single modality, failing to establish spatial correspondences between surface optical images, depth images, and X-ray images. Consequently, they cannot construct three-dimensional surface models or achieve precise cross-modal alignment and joint analysis. Summary of the Invention

[0006] To address this, the present invention provides a multimodal intelligent imaging system and its imaging method, which overcomes the problems of lack of spatial consistency between image modalities and unstable risk area identification leading to coarse and repetitive imaging planning in the prior art by establishing a multimodal collaborative three-dimensional surface dynamic verification mechanism.

[0007] To achieve the above objectives, on one hand, the present invention provides a multimodal intelligent imaging system, comprising: an information acquisition module for retrieving prior information of the user and synchronously acquiring user posture information, a first modality image set, and a second modality image set within an acquisition cycle at the same acquisition frequency; a body surface model construction module connected to the information acquisition module for constructing a three-dimensional body surface model of the user based on the user posture information and the second modality image set; an image fusion module connected to the information acquisition module and the body surface model construction module for performing a first registration based on the outline of the first modality image set and the second modality image set, and performing a second registration based on structural feature points extracted from the three-dimensional body surface model and the first modality image set; and a risk area judgment module connected to the image fusion module for... After the second registration, based on the potential risk areas obtained from prior information parsing and the newly added risk areas obtained from user posture information, the first modality image set, and the second modality image set analysis, a target risk area is determined and output based on the potential risk areas and the newly added risk areas. A region division module, connected to the risk area judgment module, is used to determine the body surface areas that need to be re-imagined and the corresponding number of re-imaginings based on the target risk areas, combined with the user posture information and historical imaging quality records, and to generate the final re-acquisition area based on the re-imagined areas. A feedback adjustment module, connected to the region division module, is used to calculate the initial acquisition frequency of the corresponding acquisition cycle for the next verification cycle based on the number of re-imaginings and the area of ​​the re-acquisition area.

[0008] Furthermore, the body surface model construction module includes a posture analysis unit and a model construction unit; the posture analysis unit is used to extract posture parameters from the user's posture information; wherein, the posture parameters include the user's joint angles, limb directions, and limb spatial position coordinates; the model construction unit is connected to the posture analysis unit and is used to construct the user's three-dimensional body surface model based on the posture parameters and the second modal image set.

[0009] Furthermore, the image fusion module includes a first registration unit and a second registration unit; the first registration unit is used to align the first modal image set and the three-dimensional body surface model to a unified reference coordinate system based on the external contours of the first modal image set and the second modal image set; the second registration unit is connected to the first registration unit and is used to extract structural feature points from the three-dimensional body surface model and the first modal image set, and determine whether there is a correspondence between the feature points of the model and the image set.

[0010] Furthermore, the second registration unit includes a feature point extraction subunit and a feature point correspondence establishment subunit; the feature point extraction subunit is used to extract structural feature points for registration from the three-dimensional body surface model and the first modal image set respectively; the feature point correspondence establishment subunit is connected to the feature point extraction subunit and is used to select the position coordinates of any two modal feature points and determine whether there is a correspondence between the two modal feature points based on the relative distance between the two modal feature points.

[0011] Furthermore, the risk area judgment module includes a prior information judgment unit and an image information judgment unit. The prior information judgment unit is used to analyze the user's past medical history records, past physical examination results, doctor annotation information, and long-term risk labels based on the user's prior information retrieved by the information acquisition module, so as to determine the potential risk areas corresponding to the user's prior information in the current verification period. The image information judgment unit is connected to the prior information judgment unit and is used to analyze the user's body surface color distribution, body surface texture structure, and body surface area shape changes in the current verification period based on the user's body posture information, the first modality image set, and the second modality image set, so as to determine the corresponding newly added risk areas in the current period.

[0012] Furthermore, the risk area judgment module also includes a risk area aggregation unit; the risk area aggregation unit is used to aggregate the prior risk areas and the newly identified risk areas in the current period, and to perform weighted fusion of the spatial range and risk level of each risk area based on the preset weights of the two to obtain the target risk area in the current verification period.

[0013] Furthermore, the region division module includes a repeat inspection planning unit and a repeat region generation unit; the repeat inspection planning unit is used to determine the body surface region that needs to be repeatedly imaged and the corresponding number of repeat imaging times within the current verification cycle based on the target risk region, combined with the user's body posture information and historical imaging quality records; the repeat region generation unit is connected to the repeat inspection planning unit and is used to generate a corresponding repeat acquisition region based on the repeatedly imaged body surface region.

[0014] Furthermore, the repeat inspection planning unit includes a repeat necessity assessment subunit and a repeat count determination subunit. The repeat necessity assessment subunit is used to assess the repeat inspection necessity of the target risk area based on the location, range, and risk level of the target risk area, and in conjunction with the blur value and artifact distribution in the historical imaging quality record. The repeat count determination subunit, connected to the repeat necessity assessment subunit, is used to determine the repeat imaging count for each area to be repeated within the current verification cycle based on the repeat inspection necessity assessment result.

[0015] Furthermore, the repeating region generation unit includes a repeating region initial construction subunit, a repeating region shape optimization subunit, and a repeating region safety constraint subunit. The repeating region initial construction subunit is used to generate an initial repeating region by expanding the boundary of its smallest circumscribed body surface partition by a preset distance based on the spatial location of the target risk region. The expansion distance is determined according to the risk level of the target risk region according to a classification rule. The repeating region shape optimization subunit is connected to the repeating region initial construction subunit and is used to smooth the boundary of the initial repeating region to ensure that the boundary of the repeating region is continuous between adjacent body surface partitions and to avoid forming isolated small areas. The repeating region safety constraint subunit is connected to the repeating region shape optimization subunit and is used to apply an area upper limit constraint and a sensitive organ exclusion constraint to the generated repeating region, and map the repeating region that meets the constraints to the corresponding spatial location of the three-dimensional body surface model and the first modality image set to output the final repeating acquisition region.

[0016] On the other hand, the present invention also provides a multimodal intelligent image photography method, comprising: retrieving prior information of the user and synchronously acquiring user body posture information, a first modality image set, and a second modality image set within an acquisition cycle at the same acquisition frequency; constructing a three-dimensional body surface model of the user based on the user body posture information and the second modality image set; performing a first registration based on the outline of the first modality image set and the second modality image set, and performing a second registration based on structural feature points extracted from the three-dimensional body surface model and the first modality image set; determining and outputting a target risk region based on the potential risk region obtained by parsing prior information and the newly added risk region obtained by analyzing the user body posture information, the first modality image set, and the second modality image set; determining the body surface region that needs to be repeatedly imaged and the corresponding number of repeated imaging based on the target risk region, combined with the user body posture information and historical imaging quality records, and generating a final repeated acquisition region based on the repeated imaging region.

[0017] Compared with existing technologies, the advantages of this invention are as follows: by constructing a multimodal intelligent imaging system that integrates X-ray images, body surface images, and three-dimensional body surface models, spatial consistency between body surface posture, external structure, and internal images is achieved, significantly improving imaging alignment accuracy; the system introduces a second registration mechanism driven by the body surface model, replacing the traditional two-dimensional contour-based registration method with local structural feature point matching, enabling multimodal images to maintain high consistency even under complex body posture conditions; the system establishes a unified risk area identification logic, achieving refined identification of body surface abnormalities through a quantitative threshold system of multi-source indicators such as prior risk labels, long-term trends, color anomalies, texture breaks, and shape mutations. The system automatically detects and identifies newly added risk areas. It proposes a hierarchical method for planning repeat examinations, using risk level, historical trends, and imaging quality records as joint criteria to dynamically calculate the necessity and number of repetitions, transforming repeat imaging from experience-based decision-making into quantifiable intelligent planning. The repeat region generation strategy constructs directly executable repeat acquisition regions through outward expansion rules, boundary smoothing, sensitive organ exclusion, and area upper limit constraints, achieving accurate and controllable repeat imaging range. This invention forms a complete automated link from risk identification to repeat region generation, improving the reliability of multimodal imaging, the targeting of repeat imaging, and the system's executability in clinical settings.

[0018] Furthermore, by using body posture analysis and 3D modeling techniques, a real-time geometric model of the user's body surface is obtained, enabling the system to accurately describe the user's body shape in a unified coordinate system. The 3D model provides a spatial reference, laying a reliable structural framework for subsequent image registration, risk localization, and region segmentation. The geometric surface established based on the real body posture effectively reduces imaging blind spots and improves the stability of subsequent multimodal matching and risk identification. Attached Figure Description

[0019] Figure 1 is a schematic diagram of the structure of the multimodal intelligent imaging system according to an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of the risk area judgment module according to an embodiment of the present invention; Figure 3 is a flowchart of the region division module according to an embodiment of the present invention; Figure 4 is a flowchart of the multimodal intelligent imaging method according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0021] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0022] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0023] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0024] Please refer to Figure 1, which is a structural schematic diagram of the multimodal intelligent imaging system and its imaging method according to an embodiment of the present invention. The present invention provides a multimodal intelligent imaging system, including: an information acquisition module, used to retrieve prior information of the user and synchronously acquire user body posture information, a first modality image set, and a second modality image set within an acquisition cycle at the same acquisition frequency; a body surface model construction module, connected to the information acquisition module, used to construct a three-dimensional body surface model of the user based on the user body posture information and the second modality image set; an image fusion module, connected to the information acquisition module and the body surface model construction module, used to perform a first registration based on the outline of the first modality image set and the second modality image set, and perform a second registration based on structural feature points extracted from the three-dimensional body surface model and the first modality image set; and a risk area judgment module. A region division module, connected to the image fusion module, is used to determine and output a target risk region based on the potential risk region obtained by parsing prior information and the newly added risk region obtained by analyzing user posture information, the first modality image set, and the second modality image set after the second registration. A region division module, connected to the risk region judgment module, is used to determine the body surface area that needs to be re-imagined and the corresponding number of re-imaginations based on the target risk region, combined with the user posture information and historical imaging quality records, and generate the final re-acquisition area based on the re-imagination area. A feedback adjustment module, connected to the region division module, is used to calculate the initial acquisition frequency of the corresponding acquisition cycle for the next verification cycle based on the number of re-imaginations and the area of ​​the re-acquisition area.

[0025] In this embodiment, prior information refers to user-related data stored by the system before the start of the current acquisition cycle. Specifically, this includes the user's past medical history records, past physical examination images, doctor annotation information, long-term risk labels, and image quality records generated by the system during historical imaging processes. The prior information, body posture parameters, image features, risk indicators, and related values ​​used in this embodiment are all data types that can be used in this technology, and do not exhaust all parameter types applicable to this invention. Prior information refers to structured or semi-structured data related to the user stored by the system before the start of the current acquisition cycle, including but not limited to: past image data, past physical examination reports and values, historical annotations, historical body posture parameters, and image quality records generated by the system. The prior information is used to assist in posture analysis, 3D body surface model construction, and preliminary estimation of potential risk areas, but is not used as direct input for the feedback adjustment module to calculate the initial acquisition frequency for the next verification cycle. The potential risk area, stored internally by the system, includes prior information for specific users, defined by medical history, previous physical examination results, historical doctor annotations, and risk tags. This information, after parsing and mapping to the current 3D body surface model coordinate space, represents one or more body surface space ranges that need to be considered in the current verification cycle. User posture information refers to data describing the user's external posture state, including joint angles, limb directions, and spatial coordinates, which can be acquired by a camera, depth camera, or inertial unit. The first modal image set refers to the data collected during the current acquisition cycle. The acquired X-ray image data may include images from multiple angles or multiple locations; the second modality image set refers to optical or depth image data used to describe the user's body surface shape and texture, including but not limited to RGB image sets, depth image sets, or three-dimensional surface scans; the three-dimensional body surface model refers to a three-dimensional geometric model reflecting the user's whole-body or partial body surface morphology, constructed based on the user's body posture information and the second modality image set, which includes surface point clouds, mesh structures, or other modeling data that can be used for spatial positioning; the first registration refers to the initial registration process of aligning the two modality images to a unified spatial reference coordinate system based on the outline or overall body posture of the two modality images, without involving high-precision feature point correspondence; the second registration refers to further extracting and matching three-dimensional body surface data. The local structural feature points of the table model and the first modality image set are used to finely correct the first registration result, so that the two modal images achieve high-precision alignment at key anatomical structures; the risk region refers to the area on the user's body surface with potential pathological risks or suspected abnormalities inferred based on prior information and image analysis results, including potential risk regions, newly added risk regions, and target risk regions fused from the two; the repeated acquisition region refers to the spatial area that needs to be repeatedly imaged, determined by the system according to the target risk region and imaging quality requirements. Its range is expanded, optimized, and constrained to guide the execution of repeated imaging; the acquisition cycle refers to a complete synchronous data acquisition process, including the entire process of user body posture information acquisition, first modality image acquisition, and second modality image acquisition;Spatial correspondence and mapping consistency refer to the one-to-one spatial correspondence between different modal images or models under a unified reference coordinate system.

[0026] This invention constructs a multimodal intelligent imaging system that integrates X-ray images, body surface images, and a 3D body surface model. This achieves spatial consistency between body surface posture, external structure, and internal images, significantly improving imaging alignment accuracy. The system introduces a second registration mechanism driven by the body surface model, replacing the traditional 2D contour-based registration method with local structural feature point matching, ensuring high consistency of multimodal images even under complex body posture conditions. The system establishes a unified risk area identification logic, using a quantitative threshold system of multiple indicators such as prior risk labels, long-term trends, color anomalies, texture breaks, and shape mutations to achieve refined detection of body surface anomalies and identify new risks. Automatic region determination; the system proposes a hierarchical repeat examination planning method, which uses risk level, historical trend and imaging quality records as joint criteria to dynamically calculate the necessity and number of repeats, transforming repeat imaging from experience-based decision-making to quantifiable intelligent planning; the repeat region generation strategy constructs directly executable repeat acquisition regions through outward expansion rules, boundary smoothing, sensitive organ exclusion and area upper limit constraints, achieving accurate and controllable repeat imaging range; this invention forms a complete automated link from risk identification to repeat region generation, improving the reliability of multimodal imaging, the targeting of repeat imaging and the executability of the system in clinical settings.

[0027] Furthermore, by using body posture analysis and 3D modeling techniques, a real-time geometric model of the user's body surface is obtained, enabling the system to accurately describe the user's body shape in a unified coordinate system. The 3D model provides a spatial reference, laying a reliable structural framework for subsequent image registration, risk localization, and region segmentation. The geometric surface established based on the real body posture effectively reduces imaging blind spots and improves the stability of subsequent multimodal matching and risk identification.

[0028] Specifically, the body surface model construction module includes a posture analysis unit and a model construction unit; the posture analysis unit is used to extract posture parameters from the user's posture information; wherein, the posture parameters include the user's joint angles, limb directions, and limb spatial position coordinates; the model construction unit is connected to the posture analysis unit and is used to construct the user's three-dimensional body surface model based on the posture parameters and the second modality image set.

[0029] In this embodiment, the posture analysis unit receives user posture information obtained by the information acquisition module and extracts the posture parameters actually used in this invention, including the rotation angles of each joint, the spatial direction vectors of each limb segment, and the position coordinates of each limb segment in three-dimensional space; the joint angles are used to describe the relative posture changes between adjacent limb segments; the limb direction vectors are used to represent the orientation of anatomical limb segments such as the trunk, upper limbs, and lower limbs; the spatial position coordinates are used to give the specific position of the center point, endpoint, or key posture node of each limb segment in the three-dimensional coordinate system; in this embodiment, the user posture information can be obtained from a structured light depth camera or a multi-view RGB camera. The machine-generated body posture estimation results are used; the body posture parameters are preprocessed, including noise smoothing, body posture continuity verification, and abnormal body posture point removal; the preprocessed body posture parameters are used as input data for the model building unit; the model building unit first establishes the basic skeletal structure of each limb of the human body in a unified coordinate system based on the body posture parameters, and performs surface fitting on the skeletal structure based on the body surface contour point cloud or depth map information extracted from the second modality image set, forming a three-dimensional mesh model corresponding to the user's real body surface shape; a body surface fitting method based on multi-view structure reconstruction or a statistical body shape model, such as the parametric regression method of SMPL, can be used.

[0030] Improving the completeness of human body posture representation and the precision of body surface reconstruction makes subsequent risk area localization, region division, and spatial mapping for image verification more accurate, thereby significantly improving the reliability and accuracy of the system in multimodal image fusion and dynamic verification.

[0031] Specifically, the image fusion module includes a first registration unit and a second registration unit; the first registration unit is used to align the first modal image set and the three-dimensional body surface model to a unified reference coordinate system based on the external contours of the first modal image set and the second modal image set; the second registration unit is connected to the first registration unit and is used to extract structural feature points from the three-dimensional body surface model and the first modal image set, and determine whether there is a correspondence between the feature points of the model and the image set.

[0032] In this embodiment, the first registration unit first extracts the outline of the user's body surface from the first modal image set, using the gray-scale abrupt change position of the body edge as the outline boundary; then, it extracts the outline of the body surface from the corresponding shooting viewpoint from the second modal image set, and maps the outline to three-dimensional space based on its depth information; after obtaining the outlines of the two modalities, the first registration unit establishes a reference coordinate system based on the three-dimensional body surface model under the reference body position, and generates a rigid transformation matrix for preliminary registration by calculating the overall geometric offset and rotation angle of the two modal outlines; the transformation matrix is ​​used to rotate and translate the first modal image set as a whole to the corresponding spatial position of the three-dimensional body surface model.

[0033] The two-level structure of first and second registration enables the images to establish a highly consistent correspondence in a unified space. The first registration establishes a preliminary alignment framework, while the second registration further locks in key structural points, achieving precise structural fusion across modalities and providing a reliable spatial consistency foundation for subsequent automatic risk analysis and repeated acquisition planning.

[0034] Specifically, the second registration unit includes a feature point extraction subunit and a feature point correspondence establishment subunit; the feature point extraction subunit is used to extract structural feature points for registration from the three-dimensional body surface model and the first modal image set respectively; the feature point correspondence establishment subunit is connected to the feature point extraction subunit and is used to select the position coordinates of any two modal feature points and determine whether there is a correspondence between the two modal feature points based on the relative distance between the two modal feature points.

[0035] In this embodiment, the feature point corresponding establishment subunit selects the local curvature change points of the body surface in the three-dimensional body surface model as candidate structural feature points. The structural feature points include surface anatomical landmarks that maintain high morphological stability under different body postures, such as the elbow tip, acromion, knee protrusion, lateral ankle point, and lumbar lateral line turning point. In the first modality image set, the contour detection algorithm based on edge intensity change and local structure template search are used to extract the candidate feature point positions corresponding to the above three-dimensional structural feature points from the two-dimensional image. In this embodiment, the neighborhood image structure of each candidate feature point is quantitatively compared. The comparison is based on a preset local structure similarity metric, including gray-level gradient direction consistency, local shape descriptor similarity, and template-based structural overlap score.

[0036] By extracting feature points and determining correspondences, high-precision matching of cross-modal structural points is achieved, establishing a consistent relationship between local anatomical landmarks of the 3D body surface model and corresponding points in the image. This mechanism improves registration accuracy, reduces structural shifts caused by modal differences, and helps to form accurate anatomical-level fusion results, providing strict spatial positioning standards for risk area judgment and repetitive area generation.

[0037] Specifically, the risk area judgment module includes a prior information judgment unit and an image information judgment unit. The prior information judgment unit is used to analyze the user's past medical history records, past physical examination results, doctor annotation information, and long-term risk labels based on the user's prior information retrieved by the information acquisition module, so as to determine the potential risk areas corresponding to the user's prior information in the current verification period. The image information judgment unit, connected to the prior information judgment unit, is used to analyze the user's body shape information, the first modality image set, and the second modality image set, to analyze the user's body surface color distribution, body surface texture structure, and body surface area shape changes in the current verification period, so as to determine the corresponding newly added risk areas in the current period.

[0038] In this embodiment, the following specific thresholds and rules are used to determine various abnormalities and trends: Continuous occurrence determination: If a body surface location is recorded as abnormal in the last three consecutive collection cycles, it is determined to be persistent; Repeated annotation determination: If a location is annotated by the doctor three or more times in the last five collection cycles, it is determined to be a repeatedly annotated location; Expanding trend determination: If the annotated area of ​​a risk area increases by more than or equal to 10% in two consecutive historical records, it is recorded as one area expansion; if this condition is met for two consecutive cycles, a stable expanding trend is determined; Color abnormality threshold: Using color brightness or hue as a comparison measure, if the local brightness change in the current cycle exceeds... If the color fluctuation range at the same location in the previous cycle is twice the normal fluctuation range, or the hue shift is greater than or equal to 10%, then the location is considered a color abnormality. Texture abnormality threshold: When a texture line that should have been continuous in the previous cycle breaks in the current cycle, and the break length reaches five millimeters or exceeds twice the average interval between adjacent normal texture lines, it is determined to be a texture breakage abnormality; where the average interval is based on the average distance between continuous texture lines extracted from the same body surface area in the previous cycle. Outer contour abnormality threshold: If a local protrusion or depression appears at a certain location in the current cycle, and its height or depth relative to the surrounding normal contour exceeds three times the surrounding normal fluctuation value, and the height or depth is greater than three millimeters, then it is determined to be an outer contour abnormality. Sudden Changes; New Risk Determination: When a segment first meets any of the above-mentioned anomaly thresholds in the current period, and no similar anomalies have appeared in the previous period or earlier historical records, the segment is determined to be a new risk area; Overlap Relationship Determination: The current new anomaly area is spatially compared with the previously inferred potential risk area in the three-dimensional surface model; if the overlap area between the new area and the potential area is greater than 80%, it is considered a complete overlap; if the overlap area is within the range [20%, 80%], it is a partial overlap; if the overlap area is less than 20%, there is no overlap; Progressive New Addition Determination: If the area of ​​the new risk area increases by more than or equal to 10% during two consecutive collection periods, it is marked as a significant increase. The system identifies newly emerging risks as progressing; if the three-dimensional coordinate offset of the same risk point in adjacent cycles is less than or equal to five millimeters, it is considered to be in a stable position and a continuation of the same risk point; if the offset exceeds five millimeters, it is considered to be possibly migrating or a new location, and should be handled according to the new rules; the above-mentioned color threshold, texture threshold, shape threshold, area threshold, and overlap ratio threshold are all empirical judgment values ​​used in this embodiment, and the system can directly use these values ​​as default thresholds when deploying according to this embodiment; if adjustments are needed in specific clinical or deployment scenarios, the above values ​​can be used as reference benchmarks, but the judgment logic described in the embodiments and claims of this patent uses these explicit thresholds as exemplary implementation schemes.

[0039] Potential and new risks are identified by a joint analysis mechanism that utilizes prior information and current periodic image features; fixed threshold color, texture, and deformation judgment rules improve the consistency and objectivity of risk identification; the module can extract multidimensional anomaly features from different modalities, making risk locking more accurate and providing a clearer basis for subsequent repeated acquisition strategies.

[0040] Referring to Figure 2, which is a structural schematic diagram of the risk area judgment module according to an embodiment of the present invention; specifically, the risk area judgment module further includes a risk area aggregation unit; the risk area aggregation unit is used to aggregate the prior risk area and the newly identified risk area in the current period, and to perform weighted fusion of the spatial range and risk level of each risk area based on the preset weights of the two to obtain the target risk area in the current verification period.

[0041] In this embodiment, after completing the weighted fusion of the prior risk area and the newly added risk area, the system first redefines the spatial boundary of the fused area. The rule for determining the spatial boundary is: the larger of the coverage areas of the two types of areas at the same body surface location is used as the basic boundary, where the larger refers to the larger area after projection of the two areas onto the body surface at that location. Based on the specific location of color change, texture change, or deformation change in the newly added risk area, the basic boundary is continuously expanded outward by two to five millimeters towards the change location, so that the final boundary can completely cover the location where the anomaly occurred within this period. In this embodiment, 3 millimeters is selected. The risk intensity calculated by the risk level of the two types of areas before fusion and their corresponding weights is used as the basis for risk assessment. The risk intensity is the numerical result obtained by summing the risk level of the potential risk area with a weight of 0.7 and the risk level of the newly added risk area with a weight of 0.3. The specific classification rule is: when the risk intensity reaches 0.7 or above, the area is determined as high risk; when the risk intensity is between 0.4 and 0.7, it is determined as medium risk; when the risk intensity is below 0.4, it is determined as low risk.

[0042] By weighting and fusing prior risks and new risks, the system can accurately reflect the true risk status of the current cycle; the fused classification rules further enhance the reliability of risk management and make subsequent repeated detection strategies more targeted.

[0043] Referring to Figure 3, which is a schematic diagram of the region division module in an embodiment of the present invention; specifically, the region division module includes a repeat inspection planning unit and a repeat region generation unit; the repeat inspection planning unit is used to determine the body surface area to be repeatedly imaged and the corresponding number of repeat imaging times within the current verification cycle based on the target risk area, combined with the user's body posture information and historical imaging quality records; the repeat region generation unit is connected to the repeat inspection planning unit and is used to generate a corresponding repeat acquisition area based on the repeatedly imaged body surface area.

[0044] In this embodiment, the repeat inspection planning unit first comprehensively evaluates the risk level, spatial location, historical change trend, and the aforementioned explicit threshold rules of each risk area based on the aforementioned target risk areas to determine the surface areas that need to be repeatedly imaged and the number of repeat imaging times within the current verification cycle. When a risk area is identified as a high-risk area, the system assigns it the highest priority and arranges at least two repeat imaging sessions in the current cycle. When an area is identified as a progressively emerging risk area, i.e., its area increases by 10% or more within two consecutive cycles, or it exhibits new color anomalies, texture breaks, or shape changes not seen in the previous cycle, this type of area is assigned a medium priority, and the system arranges at least one repeat imaging session in the current cycle. If this area also forms a partial overlap with a priori risk area... When the overlap ratio is between 20% and 80%, the number of repetitions is further increased to two. If the 3D coordinate offset of a risk point between adjacent cycles does not exceed five millimeters, the system determines that the risk point is stable and arranges one or two repetitions according to its risk level. If the offset exceeds five millimeters, it is considered a possible migration or new location and is automatically included in the repetition inspection path of the newly added risk area, and at least one repetition is arranged. For low-risk areas, no repetition is arranged. If the user has unstable imaging quality in the historical records, that is, the user's historical imaging quality records show that the same part has high imaging noise, large body deviation or local occlusion at least once in the past three cycles, the system arranges one supplementary imaging for the low-risk area according to the historical quality records.

[0045] Differentiated repetition strategies are implemented based on risk level and imaging stability to improve imaging reliability in high-risk areas while avoiding over-imaging in low-risk areas; this mechanism enhances the overall imaging efficiency and diagnostic effectiveness of the system.

[0046] Specifically, the repeat inspection planning unit includes a repeat necessity assessment subunit and a repeat count determination subunit. The repeat necessity assessment subunit is used to assess the repeat inspection necessity of the target risk area based on the location, range, and risk level of the target risk area, and in combination with the blur value and artifact distribution in the historical imaging quality record. The repeat count determination subunit is connected to the repeat necessity assessment subunit and is used to determine the repeat imaging count of each area to be repeated within the current verification cycle based on the repeat inspection necessity assessment result.

[0047] In this embodiment, the repetition necessity assessment subunit determines whether repeated imaging is necessary based on the specific location, spatial range, and risk level of the target risk area. Simultaneously, it considers the blur value, artifact frequency, and imaging stability of the most recent three acquisitions in historical imaging quality records to determine whether image quality might affect the current cycle's interpretation result for that area. If a target risk area is classified as high-risk, the system directly determines that the area needs to be re-examined. If it is medium-risk, further judgment is needed based on the area's stability in historical records: if any of the color anomalies, texture anomalies, or shape abrupt changes have occurred in the last two acquisition cycles, it is considered necessary to repeat. If it is low-risk, it is only marked as needing repeated inspection when there is a significant decrease in the historical imaging quality corresponding to the area, i.e., when the blur value in the last two acquisitions exceeds twice the normal fluctuation range, or when the artifact coverage area exceeds half the area of ​​the area. Regarding imaging quality, when the blur value of a certain body surface area exceeds the historical value of that location in the last two acquisition cycles... If the average blur value is twice that of the target area, or if obvious artifacts appear in the same area for three consecutive cycles and the artifact coverage area exceeds one-third of the area, the system determines that the location, regardless of its risk level, needs to be re-imaged. If the risk level of a region is high and the region shows significant progress in the previous cycle, including any one of the following: area growth exceeding 10%, color shift exceeding twice the normal fluctuation range of the previous cycle, or texture breakage length exceeding five millimeters, the number of re-images for that region is set to three to ensure sufficient stability in the current cycle. If it is a medium-risk region, the number of re-images is set to two. If it is a low-risk region included in the re-examination due to insufficient imaging quality, the number of re-images is set to one. When a target risk region is located in a position with large surface curvature in the 3D body surface model, which is prone to forming imaging blind spots, such as the scapular region, the lateral waist region, or the lower edge of the jaw, the system will increase the number of re-images by one in addition to the above to ensure coverage of different shooting angles and improve imaging integrity.

[0048] By combining risk level, ambiguity value and artifact status, the necessity of repetition is dynamically assessed and the optimal number of repetitions is determined, so that the system can reduce invalid repetitions while ensuring diagnostic quality, thereby improving overall image stability and risk detection rate.

[0049] Specifically, the repeating region generation unit includes a repeating region initial construction subunit, a repeating region shape optimization subunit, and a repeating region safety constraint subunit. The repeating region initial construction subunit is used to generate an initial repeating region by expanding the boundary of its smallest circumscribed body surface partition by a preset distance based on the spatial location of the target risk region. The expansion distance is determined according to the risk level of the target risk region according to a classification rule. The repeating region shape optimization subunit, connected to the repeating region initial construction subunit, is used to smooth the boundary of the initial repeating region to ensure that the boundary of the repeating region is continuous between adjacent body surface partitions and to avoid forming isolated small areas. The repeating region safety constraint subunit, connected to the repeating region shape optimization subunit, is used to apply an area upper limit constraint and a sensitive organ exclusion constraint to the generated repeating region, and to map the repeating region that meets the constraints to the corresponding spatial location of the three-dimensional body surface model and the first modality image set to output the final repeating acquisition region.

[0050] In this embodiment, the repeating region generation unit constructs the final repeating acquisition region according to the following steps: The initial construction subunit of the repeating region determines the minimum outer surface partition of the target risk region based on its spatial position in the three-dimensional body surface model; according to the risk level classification expansion rules set in this embodiment, the boundary of the minimum outer surface partition is expanded outward by a fixed distance: when the target risk region belongs to the high risk level, the expansion distance is 20 mm; when it belongs to the medium risk level, the expansion distance is 15 mm; when it belongs to the low risk level, the expansion distance is 10 mm; the initial repeating region is formed by expansion to ensure sufficient safety margin for repeating imaging coverage; if there are obvious jumps, angles, or fragmentation zones at the boundary between adjacent body surface partitions in the initial repeating region, the abnormally protruding parts are cut off and the concave parts are filled to make the boundary line maintain a natural transition on the three-dimensional curved surface; if the shape of the target risk region causes isolated small areas with an area of ​​less than 20 square millimeters to appear inside the initial repeating region, these small areas are merged with the main region or directly removed. To ensure that the final repeating region is structurally coherent and that meaningless fragments are not repeatedly acquired, the repeating region safety constraint subunit applies two types of safety restrictions to the smoothed repeating region. The first type is an area upper limit constraint: if the area of ​​a repeating region exceeds 1.5 times the average area of ​​its corresponding body surface partition, its area is compressed to the upper limit by proportionally reducing the boundary to avoid excessive expansion of the repeating region and affecting the overall repeating imaging plan of the system. The second type is a sensitive organ exclusion constraint: if the repeating region covers the chest heart projection area, the anterior thyroid gland area, the periorbital area, the genital area, or other system-preset sensitive areas, exclusion holes are formed by deducting the corresponding body surface fragments of these sensitive areas from the repeating region. If the exclusion results in unnatural boundary breaks, the shape optimization logic automatically performs small-scale smoothing again to make the excluded region coherent and usable for actual imaging. Repeating regions that meet all the above restrictions are mapped back to the corresponding spatial positions in the three-dimensional body surface model and the first modality image set by the repeating region safety constraint subunit to obtain the final repeating acquisition region.

[0051] It generates repeatable acquisition areas with reasonable structure and natural boundaries, effectively avoiding area fragmentation and unnecessary expansion. The safety constraint mechanism ensures that sensitive areas are automatically excluded and the area is strictly controlled, providing an executable spatial range for repeat imaging and improving the availability, safety and accuracy of repeat acquisition.

[0052] Specifically, in the feedback adjustment module, the formula for calculating the regional weight coefficient is as follows: Where Wi is the comprehensive weight of the i-th region, used to measure the contribution of this region to the acquisition frequency; Ni is the number of times this region is repeatedly imaged in this cycle; Nmax is the maximum number of times repeatedly imaged in all current repeated regions, used for normalization to prevent inconsistencies in magnitude between different users; Ai is the area of ​​the repeated region, in mm²; Amax is the maximum area of ​​all repeated regions, used for normalization; α is the importance weight of the number of repeated images, preferably 0.6 in this embodiment; β is the importance weight of the region area, preferably 0.4 in this embodiment; the total weight is calculated using the following formula: Wsum: The sum of weights of all repeatedly sampled regions; k: The number of repeatedly sampled regions; Among them, F init The initial sampling frequency of the next verification cycle, in Hz; F min The lowest frequency that the system can accept; preferably, in this embodiment, 0.5 is selected; F max The highest frequency that the system can accept, preferably 2.0 in this embodiment; W sum : The total weight of ; k: The number of areas collected repeatedly.

[0053] Through weight calculation and adaptive acquisition frequency adjustment mechanism, the initial acquisition frequency of the next cycle is dynamically generated based on the quantification results of repeated imaging times and area, so as to realize the automatic tilting allocation of acquisition resources to key areas, improve overall imaging efficiency, reduce invalid repeated acquisitions, and enhance the continuity and clinical usability of cross-cycle imaging data.

[0054] Referring to Figure 4, which is a flowchart of the multimodal intelligent imaging method according to an embodiment of the present invention, the present invention also provides an imaging method applied to the above-mentioned multimodal intelligent imaging system, including: Step S1: Retrieving the user's prior information and synchronously acquiring the user's body posture information, a first modality image set, and a second modality image set within the acquisition cycle at the same acquisition frequency; Step S2: Constructing a three-dimensional body surface model of the user based on the user's body posture information and the second modality image set; Step S3: Performing a first registration based on the outline of the first modality image set and the second modality image set, and performing a second registration based on the structural feature points extracted from the three-dimensional body surface model and the first modality image set; Step S4: Based on the potential risk areas obtained from prior information parsing and the newly added risk areas obtained from user posture information, the first modality image set, and the second modality image set analysis, determine and output the target risk area based on the potential risk areas and the newly added risk areas; Step S5: Based on the target risk area, combined with the user posture information and historical imaging quality records, determine the body surface area that needs to be re-imagined and the corresponding number of re-imaginings, and generate the final re-acquisition area based on the re-imagining area; Step S6: Based on the number of re-imaginings and the area of ​​the re-acquisition area, calculate the initial acquisition frequency of the acquisition cycle corresponding to the next verification cycle.

[0055] The multimodal intelligent imaging method provided in this embodiment of the invention can be applied to multimodal intelligent imaging systems and can achieve the same technical effect, which will not be elaborated here.

[0056] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal intelligent imaging system, characterized in that, include: The information acquisition module is used to retrieve the user's prior information and synchronously acquire the user's body posture information, the first modality image set, and the second modality image set within the acquisition cycle at the same acquisition frequency; A body surface model construction module, connected to the information acquisition module, is used to construct a three-dimensional body surface model of the user based on the user's body posture information and the second modality image set; The image fusion module, connected to the information acquisition module and the body surface model construction module, is used to perform a first registration based on the external contours of the first modality image set and the second modality image set, and to perform a second registration based on the structural feature points extracted from the three-dimensional body surface model and the first modality image set. The risk area determination module, connected to the image fusion module, is used to determine the risk area after the second registration. root Based on the potential risk areas obtained from prior information analysis and the newly added risk areas obtained from user posture information, the first modality image set, and the second modality image set analysis, the target risk area is determined and output according to the potential risk areas and the newly added risk areas; The region segmentation module, connected to the risk region judgment module, is used to determine the body surface region that needs to be repeatedly imaged and the corresponding number of times of repeated imaging based on the target risk region, combined with the user's body posture information and historical imaging quality records, and to generate the final repeated acquisition region based on the repeated imaging region. The feedback adjustment module, connected to the region division module, is used to calculate the initial acquisition frequency of the corresponding acquisition cycle for the next verification cycle based on the number of repeated imaging attempts and the area of ​​the repeated acquisition region.

2. The multimodal intelligent imaging system according to claim 1, characterized in that, The body surface model construction module includes a posture analysis unit and a model construction unit. The posture analysis unit is used to extract posture parameters from the user's posture information. The posture parameters include the user's joint angles, limb directions, and limb spatial coordinates. The model construction unit is connected to the posture analysis unit and is used to construct a three-dimensional body surface model of the user based on the posture parameters and the second modal image set.

3. The multimodal intelligent imaging system according to claim 1, characterized in that, The image fusion module includes a first registration unit and a second registration unit; the first registration unit is used to align the first modal image set and the three-dimensional body surface model to a unified reference coordinate system based on the external contours of the first modal image set and the second modal image set. The second registration unit, connected to the first registration unit, is used to extract structural feature points from the three-dimensional body surface model and the first modal image set, and to determine whether there is a correspondence between the feature points of the model and the image set.

4. The multimodal intelligent imaging system according to claim 3, characterized in that, The second registration unit includes a feature point extraction subunit and a feature point correspondence establishment subunit; the feature point extraction subunit is used to extract each structural feature point for registration from the three-dimensional body surface model and the first modal image set respectively; A feature point corresponding sub-unit is established and connected to the feature point extraction sub-unit to select the position coordinates of any two modal feature points, and to determine whether there is a correspondence between the two modal feature points based on the relative distance between them.

5. The multimodal intelligent imaging system according to claim 1, characterized in that, The risk area judgment module includes a prior information judgment unit and an image information judgment unit. The prior information judgment unit is used to analyze the user's past medical history records, past physical examination results, doctor annotation information and long-term risk labels based on the user's prior information retrieved by the information acquisition module, so as to determine the potential risk area corresponding to the user's prior information in the current verification period. The image information judgment unit, connected to the prior information judgment unit, is used to analyze the user's body surface color distribution, body surface texture structure, and body surface region shape changes within the current verification period based on the user's body posture information, the first modal image set, and the second modal image set, in order to determine the corresponding newly added risk areas within the current period.

6. The multimodal intelligent imaging system according to claim 5, characterized in that, The risk area judgment module also includes a risk area aggregation unit; The risk area aggregation unit is used to aggregate the prior risk areas and the newly identified risk areas in the current period, and to perform weighted fusion of the spatial range and risk level of each risk area based on the preset weights of the two to obtain the target risk area in the current verification period.

7. The multimodal intelligent imaging system according to claim 1, characterized in that, The region division module includes a repeat check planning unit and a repeat region generation unit; The repeat inspection planning unit is used to determine the body surface area that needs to be repeatedly imaged and the corresponding number of repeat imagings within the current verification cycle, based on the target risk area, combined with the user's body posture information and historical imaging quality records. The repeat region generation unit, connected to the repeat inspection planning unit, is used to generate a corresponding repeat acquisition region based on the body surface region of the repeat imaging.

8. The multimodal intelligent imaging system according to claim 7, characterized in that, The repeat inspection planning unit includes a repeat necessity assessment subunit and a repeat number determination subunit. The repeat necessity assessment subunit is used to assess the repeat inspection necessity of the target risk area based on the location, range and risk level of the target risk area, and in combination with the blur value and artifact distribution in the historical imaging quality record. The repetition count determination subunit, connected to the repetition necessity assessment subunit, is used to determine the number of re-imaging times for each region to be re-imagined within the current verification cycle based on the repetition check necessity assessment results.

9. The multimodal intelligent imaging system according to claim 7, characterized in that, The repeating region generation unit includes a repeating region initial construction subunit, a repeating region shape optimization subunit, and a repeating region safety constraint subunit. The repetitive region initial construction subunit is used to generate an initial repetitive region by expanding the boundary of its smallest circumscribed body surface partition by a preset distance based on the spatial location of the target risk region. The expansion distance is determined according to the risk level of the target risk region according to a classification rule. The repetitive region shape optimization subunit is connected to the repetitive region initial construction subunit and is used to smooth the boundary of the initial repetitive region to ensure that the boundary of the repetitive region is continuous between adjacent body surface partitions and to avoid forming isolated small areas. The repetitive region safety constraint subunit is connected to the repetitive region shape optimization subunit and is used to apply an area upper limit constraint and a sensitive organ exclusion constraint to the generated repetitive region, and map the repetitive region that meets the constraints to the corresponding spatial location of the three-dimensional body surface model and the first modality image set to output the final repetitive acquisition region.

10. A photographic method applied to the multimodal intelligent imaging system according to any one of claims 1-9, characterized in that, include: The system retrieves the user's prior information and simultaneously acquires the user's body posture information, the first modality image set, and the second modality image set within the acquisition period at the same acquisition frequency. Based on the user's body posture information and the second modal image set, a three-dimensional body surface model of the user is constructed; a first registration is performed based on the external contours of the first modal image set and the second modal image set, and a second registration is performed based on the structural feature points extracted from the three-dimensional body surface model and the first modal image set; Based on the potential risk areas obtained from prior information analysis and the newly added risk areas obtained from user body posture information, the first modality image set, and the second modality image set analysis, the target risk area is determined and output according to the potential risk areas and the newly added risk areas; Based on the target risk area, combined with the user's body posture information and historical imaging quality records, the body surface area that needs to be repeatedly imaged and the corresponding number of times to be repeatedly imaged are determined, and the final repeated acquisition area is generated based on the repeated imaging area.

Images