Psoriasis lesion evolution trajectory prediction method based on multi-site timing collaborative perception

Abstract

The application relates to a psoriasis lesion evolution track prediction method based on multi-part time sequence cooperative perception, which comprises the following steps: collecting psoriasis lesion skin images of multiple preset body parts of a patient at multiple visit time points; segmenting and extracting a target region after preprocessing each image, and calculating a visual state vector set of all body parts at each visit time point; inputting the visual state vector set at each visit time point into a spatial cooperative hypergraph encoder to output a multi-part cooperative state matrix and a hyperedge set at each visit time point; inputting the multi-part cooperative state matrix sequence and the hyperedge set at all visit time points into a cooperative evolution time sequence predictor to generate a predicted cooperative state matrix sequence at future visit time points; embedding state embedding vectors of each body part in the predicted cooperative state matrix sequence into visual index decoders to decode visual state index prediction values and generate evolution track prediction images of target regions of each body part.

Description

Technical Field

[0001] This application belongs to the field of computer vision and spatiotemporal sequence modeling technology, and in particular relates to a method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception. Background Technology

[0002] Psoriasis is a chronic, relapsing, inflammatory systemic disease. During its course, the response of skin lesions on different body parts to treatment often exhibits significant heterogeneity: lesions on the trunk and upper limbs tend to resolve more quickly, while lesions on the lower limbs, scalp, palms, and soles may remain for a long time or become stubborn, resulting in the objective phenomenon of asynchronous evolution of lesions in multiple sites within the same patient. Because this asynchronous evolution involves complex dynamic changes in multiple body parts over a long period, it is difficult to make objective, quantitative, and forward-looking predictions about the future development trends of lesions in each site based solely on manual observation and recording.

[0003] In response to the aforementioned scenarios, the automated analysis and trend prediction of multi-site skin lesion images from psoriasis patients using computer vision technology has become an objective technological need. By processing the acquired skin images, extracting the visual state features of the lesion areas, and analyzing their changes over time, it is expected to provide technical support for the objective description and trend prediction of the long-term evolution of psoriasis lesions.

[0004] Currently, computer-aided analysis techniques for skin images have made some progress, but they are mostly concentrated in the following technical paths: First, classifying or segmenting images acquired at a single time point, only assessing the skin lesion status at the current moment, and failing to capture evolutionary patterns from sequence data at multiple time points; Second, using generative models to generate simulated skin lesion appearance at a future moment based on a single image. This type of method generates synthetic images rather than quantified state trajectories, and does not involve using the historical sequence data of the same patient to model personalized evolutionary patterns.

[0005] More importantly, existing technologies generally suffer from the following limitations when dealing with multi-site and multi-time-point skin lesion image analysis scenarios in psoriasis: First, each site is treated as an independent object, lacking an explicit modeling mechanism for the spatial collaborative relationships between sites, and failing to capture the objectively existing collaborative or asynchronous evolutionary patterns between body parts such as the trunk and lower limbs, and the scalp and trunk; Second, the utilization of temporal information is mostly limited to simple comparisons of adjacent time points, failing to deeply learn the dynamic evolutionary patterns of the collaborative relationships themselves from multi-time-point sequence data; Third, there is a lack of systematic technical solutions that organically integrate the spatial collaborative relationships of multiple sites with temporal evolutionary patterns, and thereby achieve forward prediction of the future evolutionary trajectory of skin lesions in each site.

[0006] In summary, existing methods for analyzing psoriatic lesion images are limited by an isolated processing paradigm of single time points and single sites. They lack explicit modeling mechanisms for spatial synergistic relationships among multiple sites and cannot deeply learn the dynamic evolution patterns of synergistic relationships from multi-time point sequence data. Consequently, they cannot accurately predict the future evolution trajectory of lesions in each site. Currently, no effective solutions have been proposed to address these technical problems. Summary of the Invention

[0007] In view of the shortcomings of the prior art, the purpose of the invention is to provide a method for predicting the evolution trajectory of psoriatic skin lesions based on multi-site temporal collaborative perception.

[0008] The first aspect of this application proposes a method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception, comprising: Step 1: acquiring skin images of psoriasis lesions on multiple preset body parts at multiple visit time points of psoriasis patients; Step 2: preprocessing the skin images of psoriasis lesions on each body part at each visit time point, segmenting and extracting target regions in the preprocessed skin images of psoriasis lesions, and calculating visual state representation vectors based on the target regions to obtain a set of visual state vectors for all preset body parts at each visit time point; the target regions are psoriasis lesion regions that differ from the visual features of the background skin; Step 3: inputting the set of visual state vectors for each visit time point into a pre-trained empty... Step 4: The inter-body collaborative hypergraph encoder captures the binary collaborative association patterns between target regions of each body part, and outputs the multi-body collaborative state matrix and hyperedge set for each visit time point; Step 5: Input the multi-body collaborative state matrix sequence and hyperedge set of all visit time points into the pre-trained collaborative evolution time series predictor, and generate the predicted collaborative state matrix sequence for several future visit time points according to the dynamic evolution law of the collaborative relationship between target regions of each body part; Step 6: Decode the state embedding vector of each body part in the predicted collaborative state matrix sequence into the visual state index prediction value of the corresponding target region through the visual index decoder, and generate the evolution trajectory prediction map of the target region of each body part based on the visual state index prediction value.

[0009] According to a second aspect of the present disclosure, a storage medium is provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs the method described in any of the above embodiments.

[0010] According to a third aspect of the present disclosure, a system for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception is provided, comprising: an image acquisition module for acquiring skin images of psoriasis lesions on multiple preset body parts at multiple visit time points of a psoriasis patient; an image processing module for preprocessing the skin images of psoriasis lesions on each body part at each visit time point, segmenting and extracting target regions in the preprocessed skin images of psoriasis lesions, and calculating visual state representation vectors based on the target regions to obtain a set of visual state vectors for all preset body parts at each visit time point; the target regions are psoriasis lesion regions that differ from the visual features of the background skin; and an encoding module for inputting the set of visual state vectors for each visit time point into... A pre-trained spatial cooperative hypergraph encoder captures binary cooperative association patterns between target regions of various body parts, outputting a multi-part cooperative state matrix and hyperedge set for each visit time point. A prediction module inputs the multi-part cooperative state matrix sequence and hyperedge set from all visit time points into a pre-trained cooperative evolution time-series predictor, generating a predicted cooperative state matrix sequence for several future visit time points based on the dynamic evolution of cooperative relationships between target regions of various body parts. A decoding module decodes the state embedding vectors of each body part in the predicted cooperative state matrix sequence into predicted visual state index values ​​for the corresponding target regions using a visual index decoder, and generates an evolution trajectory prediction map of the target regions of each body part based on these predicted visual state index values.

[0011] According to a fourth aspect of the present disclosure, a psoriasis lesion evolution trajectory prediction system based on multi-site temporal collaborative perception is provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following steps: acquiring psoriasis lesion skin images of multiple preset body parts at multiple visit time points of a psoriasis patient; preprocessing the psoriasis lesion skin image of each body part at each visit time point, segmenting and extracting a target region in the preprocessed psoriasis lesion skin image, and calculating a visual state representation vector based on the target region to obtain a set of visual state vectors for all preset body parts at each visit time point; the target region being a psoriasis lesion region that differs from the background skin visual features; and setting the visual state vectors of each visit time point as follows: The set of visual state vectors of points is input into a pre-trained spatial cooperative hypergraph encoder to capture the binary cooperative association patterns between target regions of each body part, and outputs a multi-part cooperative state matrix and hyperedge set for each visit time point. The sequence of multi-part cooperative state matrices and hyperedge sets of all visit time points are input into a pre-trained cooperative evolution time series predictor. Based on the dynamic evolution law of the cooperative relationship between target regions of each body part, a sequence of predicted cooperative state matrices for several future visit time points is generated. The state embedding vectors of each body part in the predicted cooperative state matrix sequence are decoded into the visual state index prediction value of the corresponding target region by a visual index decoder. Based on the visual state index prediction value, an evolution trajectory prediction map of the target region of each body part is generated.

[0012] The beneficial effects of this application are as follows: (1) This application constructs a dual-stream processing architecture that combines spatial co-coding and temporal evolution prediction, integrating multi-site spatial co-coordination modeling and temporal dynamic evolution law learning into a unified framework. Among them, the spatial co-coordination hypergraph encoder is responsible for capturing the binary co-correlation patterns of each body part at a single time point, and the co-evolution temporal predictor is responsible for deep learning the dynamic evolution law of co-coordination relationships from sequence data at multiple time points and generating future prediction sequences. This dual-stream architecture breaks through the technical paradigm of existing technologies that can only perform static evaluation at a single time point or single-image appearance simulation, and for the first time realizes forward prediction from historical skin lesion image sequences of multiple sites to the future evolution trajectory of each site, providing a systematic technical solution for the objective description of the long-term evolution trend of psoriatic skin lesions.

[0013] (2) This application sets up a heterogeneous node initialization layer in the spatial collaborative hypergraph encoder, labels the part types according to the histological characteristics of different body parts, and configures an independent learnable projection matrix for each type to perform differential feature transformation. This mechanism enables thick keratinized parts such as palms and soles, hair follicle-rich parts such as scalp, nail plate-type parts such as nails, and conventional skin types such as trunk and limbs to undergo their own exclusive feature mapping before entering the graph message transmission. Thus, under the unified collaborative modeling framework, the unique visual evolution pattern of skin lesions of each part is preserved, and richer heterogeneous feature expression is provided for the subsequent capture of cross-part collaborative relationships.

[0014] (3) This application constructs anatomical adjacency subgraphs, symmetric adjacency subgraphs, and clinical co-evolutionary subgraphs by setting a multi-relationship composite adjacency message passing layer in the spatial collaborative hypergraph encoder. Figure 3 The design employs parallel subgraphs representing anatomical relationships determined by the distribution of nerve segments and blood vessels in the human body, symmetrical relationships determined by left-right symmetry, and asynchronous evolutionary relationships between sites obtained from clinical data on psoriasis. The message passing results of these multi-path subgraphs are dynamically fused using an adaptive gating mechanism. This design decouples site-specific relationship information from different sources and of different natures into independent subgraphs for separate processing, effectively addressing the technical problem in existing technologies where a single relationship assumption cannot fully describe the complex binary collaborative relationships between multiple sites, while also enhancing the model's interpretability.

[0015] (4) This application introduces a hypergraph modeling mechanism, setting up a cross-temporal hypergraph construction layer in the spatial collaborative hypergraph encoder. Spatial hyperedges connect three or more body part nodes participating in the same co-evolution mode, and temporal hyperedges connect node instances of the same part at multiple visit time points. Spatial hyperedges realize explicit modeling of high-order co-evolution modes of multiple parts, breaking through the inherent limitation of ordinary graph models that can only express binary pairwise relationships; temporal hyperedges realize the structured expression of the continuity of visual states of each part in the temporal dimension. The cross-temporal hypergraph jointly constituted by the two provides a rich input representation for the subsequent temporal prediction module that contains both high-order spatial collaborative relationships and temporal continuity structures.

[0016] (5) This application establishes a fully automated data processing pipeline from original skin lesion images to quantitative indicators and then to visualization of evolutionary trajectories. Step 2 extracts visual state indicators such as area proportion, redness, and scale texture from the image. Steps 3 and 4 perform collaborative relationship modeling and temporal prediction in a high-dimensional collaborative space. Step 5 decodes the prediction results and restores them into an intuitive visual indicator evolution trajectory diagram and predicted probability distribution. The entire process uses only objective visual quantitative indicators of the target area as the operation and output objects, and is suitable for application scenarios such as long-term monitoring and trend prediction of psoriatic lesion evolution. Attached Figure Description

[0017] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Throughout the drawings, the same reference numerals denote the same components. Obviously, the drawings described below are merely some embodiments described in this application, and those skilled in the art can obtain other drawings based on these drawings.

[0018] Figure 1 This is a hardware structure block diagram of a computing device for implementing the method described in Embodiment 1 of this disclosure; Figure 2 This is a flowchart of the method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception, as described in Embodiment 1 of this application; Figure 3 This is a schematic diagram of the overall process of the method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception as described in Embodiment 1 of this application; Figure 4 This is a schematic diagram of the structure of the psoriasis lesion evolution trajectory prediction system based on multi-site temporal collaborative perception as described in Embodiment 2 of this application; Figure 5 This is a schematic diagram of the psoriasis lesion evolution trajectory prediction system based on multi-site temporal collaborative perception as described in Embodiment 3 of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] Example 1

[0022] According to this embodiment, a method embodiment for predicting the evolution trajectory of psoriatic skin lesions based on multi-site temporal collaborative perception is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] The method embodiments provided in this example can be executed on a server or similar computing device. Figure 1 A hardware block diagram of a computing device for implementing a method for predicting the evolution trajectory of psoriatic lesions based on multi-site temporal collaborative sensing is shown. Figure 1 As shown, a computing device may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interface. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a computing device may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0024] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element in a computing device. As involved in the embodiments of this disclosure, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0025] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned application method for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0026] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the computing device's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0027] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computing device.

[0028] It should be noted here that, in some optional embodiments, the above... Figure 1 The computing device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computing devices.

[0029] Under the above operating environment, according to the first aspect of this embodiment, a method for predicting the evolution trajectory of psoriatic lesions based on multi-site temporal collaborative perception is provided. Figure 2 A flowchart illustrating the method is shown. Figure 3 The overall flow of the method is shown. (Reference) Figure 2 and Figure 3 As shown, the method includes: Step 1: Collect skin images of psoriasis lesions on multiple preset body parts at multiple visit time points for psoriasis patients.

[0030] In this embodiment of the application, when it is necessary to predict the future evolution trajectory of skin lesions on various body parts of a psoriasis patient, standardized imaging equipment can be used to collect skin images of psoriasis lesions on multiple preset body parts at multiple historical visit time points according to a preset follow-up plan. Let these multiple historical visit time points be denoted as... The total number of historical visit time points is m+1, and m+1≥3. This can be a baseline visit before the start of treatment (week 0). A follow-up visit can be scheduled for the fourth week of the initial treatment period. This could include visits at times such as the 16th week of treatment. These time points constitute a sequence of images reflecting early changes in the skin lesion state. Subsequent steps will use image data collected at these historical visit times to predict the visual state of skin lesions on various body parts at several future visit times (e.g., week 24 and week 52), thereby achieving a prospective prediction of future evolutionary trajectories. Multiple preset body parts, such as the scalp, face / neck, trunk, upper limbs, lower limbs, palms, soles, and nails—areas commonly affected by psoriatic lesions—and there should be no fewer than three sites to cover the main common sites on the patient's body.

[0031] Thus, through step 1, we can obtain the original image dataset indexed by three layers: patient p, time point t, and location loc. A single psoriasis lesion skin image is denoted as: ; in, and These represent the height and width (in pixels) of the original image of the body part, respectively, with 3 corresponding to the RGB three channels. This image dataset provides the original image data foundation for subsequent analysis of the visual state of skin lesions at multiple locations and time points, as well as prediction of evolutionary trajectories.

[0032] Step 2: Preprocess the psoriasis lesion skin images of each body part at each visit time point, segment and extract the target region in the preprocessed psoriasis lesion skin images, and calculate the visual state representation vector based on the target region to obtain the set of visual state vectors of all preset body parts at each visit time point; the target region is the psoriasis lesion region that differs from the background skin visual features.

[0033] In this embodiment of the application, for each psoriasis lesion skin image obtained in step 1 First, preprocessing is performed to eliminate non-physiological changes introduced by differences in shooting conditions between different acquisitions, resulting in standardized images. (H and W represent the unified height and width). Then, a pre-trained semantic segmentation network is used to perform pixel-by-pixel prediction on the standardized image, extracting target regions that differ significantly from the surrounding normal skin in visual features (such as color, texture, and boundaries), and obtaining the corresponding binary mask. Pixels with a value of 1 represent the target region. Next, based on the pixel values ​​of this binary mask and the normalized image, several indicators that can objectively quantify the visual state of the target region are calculated, such as the area ratio index describing the coverage, the redness index describing color features, and the scale texture index describing surface roughness. These indicators combined constitute the visual state representation vector of the body part at the time of the visit. (i is the location index, (where d is the feature dimension and d is the time point of the visit). Finally, the visual state representation vectors of all N preset body parts at the current visit time point are summarized to form the visual state vector set for that visit time point. .

[0034] Thus, through step 2, the skin lesion images of each site at each visit time point are transformed into a structured, quantified set of visual state vectors. This provides standardized input data for subsequent steps to model the collaborative relationships of multiple parts in a unified numerical space.

[0035] Step 3: Input the set of visual state vectors at each visit time point into the pre-trained spatial cooperative hypergraph encoder to capture the binary cooperative association patterns between target regions of each body part, and output the multi-part cooperative state matrix and hyperedge set at each visit time point.

[0036] In this embodiment, the spatial cooperative hypergraph encoder is a pre-trained neural network model with two core functions: first, capturing binary cooperative association patterns between target regions of different body parts at the same time point; and second, constructing a high-order hyperedge structure across time sequences. For the visual state vector set obtained in step 2 for each visit time point... The encoder is invoked, which first maps each body part to nodes in a graph structure and establishes connections between nodes based on domain knowledge and a data-driven approach to express binary collaborative relationships between parts in terms of anatomy, symmetry, or clinical co-evolution. Then, through message passing operations between nodes, the node representation of each part absorbs contextual information from its associated parts, thus updating into an embedding vector that incorporates collaborative information. The updated embedding vectors of all parts constitute the multi-part collaborative state matrix at the visit time point. (D represents the embedding dimension). Furthermore, based on the mined cross-time point co-evolution patterns, the encoder constructs high-order hyperedges at the current time point that can connect multiple nodes. This captures collaborative patterns involving three or more parts and the temporal continuity of the same part across different time points, forming a set of hyperedges. Output them together.

[0037] Thus, through step 3, at each historical visit time point, a multi-site collaborative state matrix incorporating the inter-site collaborative context is obtained. Furthermore, a set of hyperedges that embodies higher-order collaboration and cross-temporal association was obtained. This provides rich spatial co-representations for subsequent time series prediction.

[0038] Step 4: Input the multi-site co-state matrix sequence and hyperedge set of all visit time points into the pre-trained co-evolution time series predictor. Based on the dynamic evolution law of the co-relationship between target areas of each body part, generate the predicted co-state matrix sequence for several future visit time points.

[0039] In this embodiment, the co-evolutionary time series predictor is another pre-trained neural network model that has the function of learning the dynamic evolution law of co-evolutionary relationships from time series data and performing forward extrapolation prediction. It uses the historical visit time points obtained in step 3. arrive Cooperative state matrix sequence and the corresponding set of hyperedges As input, the predictor first enhances the cooperative state matrix at each time point using higher-order relational information from the hyperedge set. Then, through an internal temporal modeling network, it learns the dynamic evolution of the cooperative states of each part over time from the enhanced matrix sequence, generating a latent evolution vector characterizing the entire historical evolution process. Finally, based on this latent evolution vector, a forward inference network directly generates multiple consecutive future visit time points. The sequence of predicted cooperative state matrices (where L is the prediction step size, e.g., predicting week 24 and week 52) is denoted as... Each of them The format is consistent with the cooperative state matrix output in step 3.

[0040] Thus, step 4 enables forward prediction from historical multi-site skin lesion image sequences to the collaborative state at various future time points, providing core predictive data for subsequent visualization analysis.

[0041] Step 5: Decode the state embedding vectors of each body part in the predicted co-state matrix sequence into the visual state index prediction values ​​of the corresponding target regions through the visual index decoder, and generate the evolution trajectory prediction map of the target regions of each body part based on the visual state index prediction values.

[0042] In this embodiment, the visual index decoder is used to restore the high-dimensional cooperative state representation predicted in step 4 to a physically interpretable visual index. For each time point in the predicted cooperative state matrix sequence... State embedding vector for each body part The decoder maps it to a predicted value consistent with the dimension of the visual state representation vector in step 2, for example, obtaining the predicted area proportion index. Predicted Redness Index and predicting the scale texture index Subsequently, the actual observed index values ​​of each body part at each historical visit time point (from step 2) were... By connecting the predicted values ​​of the indicators at various future prediction points on a time axis, the evolution trajectory curve of the visual state indicators of each body part's target area can be plotted. Different body parts can be distinguished using different colors or line types, and confidence intervals can be simultaneously presented in the graph.

[0043] Thus, through step 5, the abstract collaborative state prediction results are decoded into an intuitive, quantitative visual state indicator evolution trajectory prediction map, completing the fully automated analysis from the original image sequence to the long-term trend visualization output.

[0044] As described in the background section, existing methods for analyzing psoriasis lesion images are limited by an isolated processing paradigm of single time point and single location. They lack explicit modeling mechanisms for spatial synergistic relationships between multiple locations and cannot deeply learn the dynamic evolution of synergistic relationships from sequence data at multiple time points. As a result, they cannot accurately predict the future evolution trajectory of lesions in each location.

[0045] In view of this, this application constructs a dual-stream processing architecture of a spatial collaborative hypergraph encoder and a co-evolutionary temporal predictor, organically integrating multi-site spatial collaborative relationship modeling with temporal evolution law learning. Specifically, the spatial collaborative hypergraph encoder captures binary collaborative association patterns between target regions of various body parts at each visit time point, outputting a multi-site collaborative state matrix and hyperedge set that integrates collaborative context; the co-evolutionary temporal predictor learns the dynamic evolution law of collaborative relationships from state sequences at multiple time points and generates predicted collaborative state sequences for future time points; finally, the visual index decoder restores the prediction results into an intuitive visual state index evolution trajectory prediction map. Through the above systematic technical solution, the technical problem of difficulty in achieving forward-looking prediction of long-term skin lesion trends due to the lack of multi-site collaborative modeling and temporal evolution prediction in existing technologies is effectively solved.

[0046] It should be noted that the "spatial collaborative hypergraph encoder" and "co-evolutionary temporal predictor" used in the embodiments of this application are both pre-trained neural network models. Their training process can be completed offline using a large amount of multi-timepoint, multi-site skin lesion image data from psoriasis patients. After training, the model parameters are fixed and directly loaded for use in the application phase. Specific training methods and the internal structure of the network will be described in detail below.

[0047] Optionally, step 2 specifically includes: performing keypoint localization and affine transformation on the psoriatic skin lesion images of each body part using a keypoint detection network, aligning the images to a standard pose and scale to obtain a standardized image; using a semantic segmentation network to perform pixel-by-pixel prediction on the standardized image, extracting target regions that differ from the visual features of the background skin, and generating a binary mask; based on the pixel values ​​of the binary mask and the standardized image, calculating the visual state representation vector of the target region for each body part, the visual state representation vector including at least one of the following indicators: area ratio index, representing the proportion of the area of ​​the target region to the total area of ​​the standardized image of the body part; redness index, representing the ratio of the intensity of the red channel to the intensity of the green and blue channels in the target region; scale texture index, representing the texture roughness statistics calculated based on the local binary pattern or gray-level co-occurrence matrix in the target region; summarizing the visual state representation vectors of all preset body parts at each visit time point to form a visual state vector set for each visit time point.

[0048] In this embodiment of the application, for each psoriasis lesion skin image obtained in step 1 Step 2 can be divided into four stages: image preprocessing, target region segmentation, visual state vector calculation, and vector set aggregation.

[0049] In the image preprocessing stage, pre-trained human keypoint detection networks (such as human pose estimation models based on Hourglass or HRNet architectures) are used to process psoriatic skin lesion images. Key point localization is performed to obtain the coordinates of pre-defined anatomical landmarks in the image. These key points can be anatomical landmarks common to different body parts, such as joint transitions, bony prominences, or intersections of specific skin folds. Based on the detected key point coordinates, an affine transformation is performed to map the original image to a predefined standard pose template, thereby aligning images of the same body part acquired at different visit times and under different shooting conditions to a uniform pose and scale. After the above processing, a standardized image is output. Where H and W are the standardized image dimensions (e.g., H=W=512 pixels), and 3 corresponds to the RGB three channels. Image standardization effectively eliminates non-physiological appearance differences introduced by changes in shooting distance, angle, or patient position, ensuring the spatiotemporal consistency and comparability of subsequent target region segmentation and visual state index calculation.

[0050] In the target region segmentation stage, a pre-trained semantic segmentation network (such as U-Net, DeepLabV3+, etc.) is used to process the standardized image. Pixel-by-pixel prediction is performed. During offline training, this semantic segmentation network learns from a large number of psoriasis lesion skin images and their corresponding target region annotations, gaining the ability to identify and extract psoriasis lesion regions. In the application phase, for the input standardized image, the semantic segmentation network outputs the probability value of each pixel belonging to the target region. After binarization processing with a preset threshold (e.g., 0.5), a binary mask is generated. In this binary mask, pixels with a value of 1 correspond to the target region, which is the psoriatic lesion area that differs significantly from the surrounding normal background skin in visual features (such as reddish color, rough texture, and clear boundaries); pixels with a value of 0 correspond to the background skin. It should be noted that this segmentation process divides regions solely based on the visual feature differences of image pixels. The extracted target region is a set of pixel coordinates and does not involve medical diagnosis or pathological classification of the lesion's nature.

[0051] In the visual state vector calculation stage, based on a binary mask and standardized images The pixel values ​​are used to calculate a set of purely visual quantization indicators within the pixel range of the target area, which constitute the visual state representation vector of that body part at the time of the visit. (i is the index of the body part, i=1,2,...,N). This visual state representation vector may include at least one of the following indices: Area ratio index This is used to characterize the proportion of a target region within a standardized image of that body part. Its calculation formula is: ; in, This represents the total number of pixels with a value of 1 in the binary mask, where H×W is the total number of pixels in the normalized image. Area percentage index. The value range is [0,1]: a value of 0 indicates that no target area was detected for that body part, and a value of 1 indicates that the target area covers the entire standardized image area of ​​that body part. This indicator quantifies the visual state of the target area from the spatial coverage dimension, and its changing trend reflects the expansion or contraction of the lesion area.

[0052] Redness Index This value represents the ratio of the red channel color intensity to the green and blue channels within a target area, reflecting the salience of the erythema in that area. The calculation formula is as follows: ; in, , , Standardized images Red, green, and blue channels This indicates a pixel-by-pixel mask multiplication operation, which retains only the channel values ​​of pixels within the target area, while the background area is not included in the calculation; This represents the arithmetic mean of the channel values ​​of all pixels within the target area after masking. For a very small positive number (e.g.) This is used to prevent the abnormal situation where the denominator is zero. Redness Index The value ranges from positive real numbers, with larger values ​​indicating a more reddish overall color in the target area. Erythema is a core feature in the visual presentation of psoriatic lesions, and this index quantifies the degree of inflammatory response in the target area from a color perspective.

[0053] Scale texture index This is used to characterize the roughness of the skin surface texture within a target area, reflecting the visual appearance of scale accumulation on the surface of psoriatic lesions. The calculation process is as follows: first, local binary pattern features or gray-level co-occurrence matrix features are extracted within the target area to construct a texture feature histogram. The histogram statistically analyzes the frequency of different local texture patterns within the target region; then, it performs a weighted summation of the statistics representing texture uniformity, contrast, directionality, and other dimensions in the histogram; finally, it normalizes the output using the Sigmoid function, mapping it to the [0,1] interval. ; in, A trainable texture weight vector, whose dimension is the same as that of the texture feature histogram. The dimensions are consistent; It is a trainable bias scalar; The activation function is Sigmoid. The scaling texture index is... The closer the value is to 1, the rougher the surface texture and the more prominent the scales in the target area; the closer the value is to 0, the smoother the surface of the target area, tending towards normal skin texture.

[0054] The calculated pure visual state indicators are combined to form the visual state representation vector of the body part at the time of the visit. For example, when all three indicators are used simultaneously, the visual state representation vector can be expressed as: ; Where d=3 represents the total feature dimension. It should be understood that the above indicators are merely illustrative; in practical applications, other quantifiable purely visual descriptors can be added as needed, such as the irregularity of the target region boundary and the contrast with the surrounding skin, to more comprehensively characterize the visual state of the target region. The calculation of each indicator is directly based on image pixel values ​​and does not rely on any clinical rating scales or medical diagnostic rules.

[0055] In the vector set aggregation stage, the visual state representation vectors calculated from all N preset body parts at the current visit time t are aggregated to form the visual state vector set for that visit time point: ; Each element in the set A quantitative description of the visual state of a target area corresponding to a specific body part at the time of the visit.

[0056] Thus, through the detailed operations of step 2 above, the skin images of psoriatic lesions on each body part at each visit time point are transformed into a set of structured visual state vectors presented in the form of purely visual quantitative indicators. This vector set has a unified format and clear semantics, providing standardized input data for the subsequent step 3 to model the spatial coordination relationship between target regions of various body parts in a unified numerical space.

[0057] Optionally, the spatial collaborative hypergraph encoder includes a heterogeneous node initialization layer, a multi-relationship composite adjacency message passing layer, and a cross-temporal hypergraph construction layer; and step 3 specifically includes: in the heterogeneous node initialization layer, based on the visual state vector set at each visit time point, constructing each body part as a graph node, labeling the part type according to the histological characteristics of each body part, and using the independent learnable projection matrix corresponding to each part type to map the visual state representation vector of each body part to the node initial embedding vector; wherein, the part type labels include: thick keratinized parts, hair follicle-rich parts, nail-like parts, and regular skin parts; in the multi-relationship composite adjacency message passing layer, constructing multiple parallel relation subgraphs, each corresponding to a different body part. The binary association type between nodes; based on the initial embedding vector of the node, message passing of the heterogeneous graph attention network is executed independently in each relational subgraph. The node update results of the multi-path relational subgraphs are fused through an adaptive gating mechanism to obtain the binary collaborative perception embedding vector of each body part, which constitutes the multi-part collaborative state matrix at the current visit time point; in the cross-temporal hypergraph construction layer, based on the difference between the binary collaborative perception embedding vector of each graph node at the current visit time point and the previous visit time point, combined with the multi-part co-evolution pattern mined from historical psoriasis patient data, a spatial hyperedge connecting at least three graph nodes and a temporal hyperedge connecting the nodes corresponding to the same graph node at multiple visit time points are constructed to form a hyperedge set for use by the co-evolutionary temporal predictor.

[0058] In this embodiment, the spatial cooperative hypergraph encoder is a pre-trained neural network model, which is responsible for processing the visual state vector set at each visit time point in step 3. Cooperative encoding of the spatial dimension is performed. The encoder consists of three functional layers stacked sequentially: a heterogeneous node initialization layer, a multi-relationship composite adjacency message passing layer, and a cross-temporal hypergraph construction layer. The functions and specific operations of each layer are described below.

[0059] In the heterogeneous node initialization layer, the encoder receives the set of visual state vectors at the current visit time t, output from step 2. The N preset body parts in the set are each represented as N nodes in a graph structure. Let the graph node corresponding to the i-th body part be denoted as . Considering the inherent differences in skin histological characteristics across different body parts—for example, the stratum corneum of the epidermis is significantly thicker on the palms and soles than in other areas, the scalp has a higher density of hair follicles and more sebum secretion, the nail area is densely keratinized hard tissue, while the trunk and limbs have conventional skin types—these differences in histological background fundamentally affect the visual appearance and evolutionary patterns of skin lesions. Therefore, this embodiment, based on the histological characteristics of each body part, assigns each graph node... Assign a part type label Its value comes from a preset set of body part types. These correspond to areas with thick keratin (such as palms and soles), areas with abundant hair follicles (such as scalp), areas with nail plates (such as nails), and areas with normal skin (such as trunk, upper limbs, and lower limbs). For each area type... Each heterogeneous node initialization layer maintains an independent learnable projection matrix. , where d is the dimension of the input visual state representation vector, and d′ is the dimension of the initial embedding vector of the projected nodes (e.g., d′=64). For each graph node Using the projection matrix corresponding to its body part type, the visual state representation vector of that body part is generated. Perform a linear projection to obtain the initial embedding vector of the node: ; The significance of using different types of dedicated projection matrices is that they allow body parts with different histological backgrounds to pass through their own independent feature transformation channels before entering the subsequent graph message transmission. This preserves the unique visual evolutionary pattern characteristics of each part within a unified collaborative modeling framework, while also providing an initial representation rich in part type semantics for cross-part information interaction.

[0060] Furthermore, the aforementioned "maintenance" refers to the existence, storage, and updating of the projection matrix as an independent parameter within the model. Specifically, during the model construction phase, four identical projection matrices are created for each of the four part types. , , and Each matrix is ​​a d'×d two-dimensional parameter tensor. During model parameter initialization, the elements of each projection matrix can be assigned values ​​using common parameter initialization strategies in deep learning (such as Xavier initialization or He initialization), allowing each matrix to start training from different initial parameter states. During the model forward propagation phase, when it is necessary to represent the visual state vector of a certain body part... When mapping to the initial embedding vector of a node, the graph node corresponding to that body part is read first. Part type label Based on the label, the corresponding projection matrix is ​​selected from four projection matrices, and then a linear projection operation of matrix multiplication is performed. During the model backpropagation training phase, the parameter values ​​of each projection matrix are independently updated according to the backpropagation of its gradient based on the loss function. That is, each projection matrix is ​​only updated by the training samples of those graph nodes belonging to the corresponding body part type. The projection matrices of different body part types maintain parameter independence and do not interfere with each other during the learning process. Through this maintenance method, each body part type has a dedicated feature transformation channel, allowing body parts with different histological characteristics such as thick keratin, abundant hair follicles, nail plates, and regular skin to undergo their own differentiated feature mapping before entering subsequent graph message passing. This effectively preserves the unique visual evolutionary pattern features of each body part within a unified collaborative modeling framework.

[0061] In the multi-relational composite adjacency message passing layer, the encoder is based on the node initial embedding vector output by the heterogeneous node initialization layer. The layer constructs a graph structure and performs message passing operations between nodes to capture binary collaborative association patterns between target regions of various body parts. The core design of this layer involves constructing multiple parallel relational subgraphs, each corresponding to a different type of binary association between body parts, thereby decoupling and modeling spatial relationships between parts of different origins and natures. In this embodiment, the relational subgraphs include the following three types: First, anatomical adjacency subgraphs, whose edges are constructed based on prior anatomical knowledge of the nerve innervation segments or blood vessel distribution areas of the human body. Connections are established between anatomically closely related body parts (e.g., the trunk and upper limb innervated by the same spinal nerve segment, the lower limb and buttock innervated by the lumbosacral nerve, etc.), and the edge weights are initialized by the normalized value of the anatomical association strength; Second, symmetrical adjacency subgraphs, whose edges are constructed based on the correspondence between symmetrical body parts (e.g., left upper limb and right upper limb, left lower leg and right lower leg). (etc.) Establish connecting edges between symmetrical body parts, with edge weights initialized by the degree of synchronicity of symmetrical parts in the evolution of psoriatic lesions; Third, the clinical co-evolution subgraph, whose edges are constructed based on the asynchronous evolution patterns of lesions between parts obtained from historical clinical data of psoriasis patients (e.g., the positive correlation between the regression of trunk lesions and the continued coexistence of lower leg lesions, the temporal correlation between scalp lesions and trunk lesions, etc.), establish connecting edges between body parts that have clinical synergy or asynchronous correlation, with edge weights obtained from the statistical correlation pre-training of clinical data.

[0062] In each relational subgraph, a message-passing operation of a heterogeneous graph attention network (such as GAT or its variant GATv2) is performed independently. This operation uses the initial embedding vector of each node as its initial state, and calculates the attention weights between the node and its neighbors in that subgraph to determine the contribution of neighboring node information to the current node. After multiple layers of message-passing iterations, each subgraph outputs a set of node update embeddings that fuse the association type information corresponding to that subgraph. Subsequently, the node update results of the three relational subgraphs are fed into an adaptive gated fusion mechanism: this mechanism first performs global average pooling on the node update results of each subgraph, inputs the pooled vectors into a lightweight multilayer perceptron, normalizes them using the Softmax function, and outputs the fusion weight coefficients corresponding to each relational subgraph. Based on these fusion weight coefficients, the node update results of each subgraph are weighted and summed to obtain the final binary collaborative perception embedding vectors for each body part. ,in For collaborative perception embedding dimension, K is the number of attention heads. (Hidden dimension for each head). The adaptive nature of the fusion weight coefficients allows the model to dynamically adjust the relative contributions of anatomical prior information, symmetric association information, and clinical co-evolutionary information to the final collaborative state representation based on the actual data characteristics of different patients. The binary collaborative perception embedding vectors of all N body parts are arranged in rows to form the multi-site collaborative state matrix at the current visit time t: ; Each row of this matrix carries the embedded representation of the corresponding body part after it has absorbed the collaborative context information of other related parts. It is the core data carrier for subsequent temporal modeling and hypergraph construction.

[0063] In the cross-temporal hypergraph construction layer, the encoder further constructs the hypergraph structure to overcome the limitation of ordinary graph models that can only express binary pairwise relationships. This enables higher-order collaborative patterns involving multiple body parts and explicit modeling of the temporal continuity of visual states of the same body part. The operations in this layer include two stages: co-evolutionary pattern mining and hyperedge construction. Co-evolutionary pattern mining is completed during the offline training phase. Specifically, from historical psoriasis patient training data, the differences in binary collaborative perception embedding vectors between every two adjacent visit time points of each graph node are extracted, forming multiple difference vectors. These difference vectors are then subjected to unsupervised clustering (e.g., spectral clustering or K-means clustering), and each cluster center is defined as a multi-body co-evolutionary pattern, with each co-evolutionary pattern corresponding to a hyperedge type. In the online inference phase, the hyperedge construction operation includes: for the current visit time point t, calculating the binary collaborative perception embedding vectors of each graph node. Binary co-sensory embedding vector with the previous visit time point t-1 The differences between them are used to obtain the current difference vector; this current difference vector is then matched with the cluster centers of each co-evolutionary pattern mined during the training phase to determine the co-evolutionary pattern to which it belongs. Based on the identified co-evolutionary patterns, a spatial hyperedge is constructed. This hyperedge connects at least three graph nodes that participate in the co-evolution mode at the same visit time point, forming a hyperedge that can express that this part of the body is currently in the same high-order collaborative state. Simultaneously, a temporal hyperedge is constructed. This hyperedge connects the node instances corresponding to the same graph node at various historical visit times, forming a hyperedge along the time axis that runs through multiple time points representing the body part. This modeles the temporal continuity and gradual change of the visual state of the target area of ​​the body part. All constructed spatial and temporal hyperedges are aggregated to form the hyperedge set for the current visit time point. .

[0064] Thus, through the sequential processing of step 3 and its three functional levels, the spatial cooperative hypergraph encoder simultaneously outputs two cooperative representations at each historical visit time point t: one is a multi-site cooperative state matrix incorporating binary cooperative context information between sites. Secondly, it captured the set of super-edges for high-order multi-part co-evolution modes and cross-temporal continuity relationships. Together, these two elements constitute the spatial co-input representation required for subsequent co-evolutionary time series predictors to perform time series modeling and forward prediction.

[0065] The training process of the spatial collaborative hypergraph encoder in this embodiment is described below.

[0066] The training of the spatial collaborative hypergraph encoder was completed offline. The training data came from a training dataset consisting of skin images of psoriasis lesions from multiple time points and locations of a large number of psoriasis patients. For each patient in the training dataset, images of multiple locations were extracted from multiple historical visit time points. The visual state vector set at each time point was calculated according to the processing flow in step 2, and the sequence data of multiple consecutive time points was used as a training sample.

[0067] During training, the spatial cooperative hypergraph encoder and the subsequent co-evolutionary temporal predictor can be jointly trained end-to-end. Specifically, for each training sample, the spatial cooperative hypergraph encoder processes the visual state vector set at each historical time point according to the three-layer processing flow described above. Encode the data and output the corresponding multi-part cooperative state matrix. and superedge set Subsequently, the co-evolutionary time series predictor receives a sequence of co-state matrices and a set of hyperedges from multiple consecutive historical time points, generating a sequence of predicted co-state matrices for several future time points. Finally, the predicted results are compared with the actual co-state matrices for future time points, a loss function (e.g., mean squared error loss) is calculated, and all trainable parameters of the encoder and predictor, including the projection matrices corresponding to each part type, are synchronously updated through the backpropagation algorithm. The weights and attention vectors of heterogeneous graph attention networks, the parameters of multilayer perceptrons in adaptive gating fusion mechanisms, and the relevant parameters for hyperedge construction in cross-temporal hypergraph construction layers are all included.

[0068] The unsupervised clustering in the co-evolution pattern mining operation described above is pre-computed using the difference vectors of adjacent time points of all patients in the training dataset before joint training begins, resulting in fixed co-evolution pattern cluster centers; these cluster centers remain unchanged during training. Joint training proceeds through multiple iterations until the loss function converges, ultimately yielding the trained spatial co-evolution hypergraph encoder model parameters. After training, the model parameter values ​​are fixed and saved, and can be directly loaded and used in the application phase without further adjustment.

[0069] Optionally, in the multi-relational composite adjacency message passing layer, the operation of constructing multiple parallel relational subgraphs includes: constructing an anatomical adjacency subgraph based on anatomical prior knowledge of human nerve innervation segments or blood vessel distribution areas; in the anatomical adjacency subgraph, the edges between nodes are defined by body parts with anatomical associations determined according to anatomical prior knowledge, and the weights of the edges are initialized by the strength of the anatomical associations; constructing a symmetrical adjacency subgraph based on the correspondence between symmetrical body parts; in the symmetrical adjacency subgraph, the edges between nodes are defined by body parts with symmetrical associations determined according to the correspondences, and the weights of the edges are initialized by the degree of synchronicity of symmetrical parts in the evolution of psoriasis lesions; constructing a clinical co-evolution subgraph based on the asynchronous evolution law of lesions between sites obtained from statistical analysis of psoriasis clinical data; in the clinical co-evolution subgraph, the edges between nodes are defined by body parts with clinical synergy or asynchronous associations determined according to the asynchronous evolution law, and the weights of the edges are obtained by pre-training based on the statistical correlation of clinical data.

[0070] In this embodiment of the application, the operation of constructing multiple parallel relation subgraphs in the multi-relation composite adjacency message passing layer is implemented through the following three sub-steps. Each subgraph is structurally independent and processed in parallel in terms of information flow, respectively capturing binary association relationships between body parts of different natures.

[0071] First, an anatomical adjacency subgraph is constructed. This subgraph is built upon prior anatomical knowledge of the segments innervated by nerves or the distribution areas of blood vessels in the human body. Specifically, the innervation of human skin exhibits a segmental distribution pattern: the cutaneous branches of adjacent spinal nerves form continuous band-like innervation areas on the body surface. For example, the C5 to C7 nerve segments mainly innervate the extensor surfaces of the upper limbs and part of the upper trunk; the T8 to T12 nerve segments mainly innervate the lower middle trunk and abdomen; the L2 to L4 nerve segments mainly innervate the anterior and medial parts of the lower limbs; and the S1 to S3 nerve segments mainly innervate the posterior parts of the lower limbs and the buttocks. Simultaneously, the arterial blood supply to the skin also follows a regional distribution pattern, with specific vascular trunks and their branches responsible for supplying blood to specific areas of the body surface. Based on this prior anatomical knowledge, body parts located within the same nerve innervation segment or the same blood vessel distribution area are identified as having an anatomical relationship, and an edge is established between the corresponding two graph nodes. For example, in the anatomical adjacency subgraph, the upper limb and upper trunk are connected by an edge due to their shared innervation by cervical nerve segments; the lower limb and buttocks are connected by an edge due to their shared innervation by lumbosacral nerve segments; and the scalp and face / neck are connected by an edge due to the shared distribution of the cervical plexus and external carotid artery branches. The initial weights of the edges are determined by the strength of the anatomical association: the association strength can be quantified and graded according to the degree of overlap in nerve innervation or the degree of co-occurrence of vascular distribution. For example, pairs of locations sharing the same nerve segment are assigned higher initial weights, while pairs of locations that are only indirectly associated by adjacent nerve segments are assigned lower initial weights. The edge weights in the anatomical adjacency subgraph can be further fine-tuned during network training to adaptively optimize the expression of association strength.

[0072] Second, a symmetrical adjacency subgraph is constructed. This subgraph is built based on the correspondence between symmetrical parts of the body. The distribution of psoriatic lesions on both sides of the body often exhibits certain symmetrical characteristics: in some patients, symmetrical areas such as the left and right upper limbs, left and right lower legs, and left and right palmar-plantar surfaces may show lesions simultaneously or exhibit similar trends during treatment. Based on this pattern, body parts that are mirror images of each other on either side of the body's midline are identified as having a symmetrical relationship, and an edge is established between the corresponding two graph nodes. For example, an edge is established between the left and right upper limbs, the left and right lower limbs, and the left and right palmar-plantar surfaces. For midline areas such as the face / neck, relationships with themselves or other midline structures can be set according to actual needs. The initial edge weights are determined by the degree of synchronicity of symmetrical sites in the evolution of psoriatic lesions: the degree of synchronicity can be quantified by statistically analyzing the consistency of changes in lesion scores at symmetrical sites in historical clinical data; sites with higher synchronicity are assigned higher initial edge weights. The existence of symmetrical adjacency subgraphs allows subsequent message passing operations to naturally utilize the universally applicable biomedical prior of left-right symmetry, enhancing the rationality and stability of collaborative state modeling.

[0073] Third, a clinical co-evolution subgraph is constructed. This subgraph is built based on the asynchronous evolution patterns of skin lesions between different locations, statistically obtained from historical clinical data of psoriasis patients. Unlike the previous two subgraphs, which rely on prior medical knowledge, the edge structure of the clinical co-evolution subgraph is determined entirely by data. Specifically, during the offline pre-training phase, a large amount of multi-timepoint, multi-location skin lesion data from psoriasis patients is used to perform statistical analysis and hypothesis testing on the correlation of skin lesion state changes between any two different body parts. For body part pairs that show a statistically significant positive correlation (i.e., the aggravation or improvement of skin lesions in one location occurs synchronously with the aggravation or improvement of skin lesions in another location) or a significant negative correlation (i.e., the improvement of skin lesions in one location is accompanied by the aggravation of skin lesions in another location, exhibiting an asynchronous pattern of one increasing and the other decreasing), their clinical synergy or asynchronous association is confirmed, and an edge connection is established between the corresponding two graph nodes. For example, real-world research data shows that skin lesions on the trunk and upper limbs tend to heal quickly, while lesions on the lower limbs, especially the calves, often heal slowly or tend to be stubborn. Statistically, this pattern manifests as a stable negative correlation between changes in trunk and calf skin lesions, thus establishing an edge connection between the trunk and calves. Similarly, the existence of edge connections between the scalp and trunk, upper and lower limbs, and palms, soles, and nails can be determined based on statistical correlation analysis. The edge weights are obtained through pre-training based on the statistical correlation coefficient: the higher the correlation coefficient (regardless of whether it is positive or negative, its absolute value is used to measure the strength of the association), the larger the initial edge weight. After pre-training, the edge weights of the clinical co-evolution subgraph are saved as part of the model parameters and can be further fine-tuned in subsequent end-to-end joint training. It should be noted that the edge structure of the clinical co-evolution subgraph may vary from person to person. That is, for different types of psoriasis patient groups (such as patients of different ages, different disease courses, and different severity), the statistically presented inter-site association patterns may differ. Therefore, with sufficient training data, customized clinical co-evolution subgraph structures can be learned for different patient subgroups.

[0074] Thus, through the parallel operation of the above three sub-steps, the multi-relation composite adjacency message passing layer obtains three relational subgraphs with distinct structures and complementary relational sources: the anatomical adjacency subgraph provides hard spatial associations based on the inherent anatomical structure of the human body, the symmetric adjacency subgraph provides regular associations based on body symmetry, and the clinical co-evolution subgraph provides statistical associations based on real clinical data. These three parallel relational subgraphs describe the possible binary collaborative or asynchronous relationships between body parts from different dimensions, laying the structural foundation for subsequent independent execution of message passing in each subgraph and the fusion of multi-path information through an adaptive gating mechanism.

[0075] Optionally, spatial and temporal hyperedges are constructed through the following operations: From historical psoriasis patient data, the differences in binary co-perceptual embedding vectors of each graph node at adjacent visit time points are extracted to obtain multiple difference vectors; unsupervised clustering is performed on these multiple difference vectors to obtain several cluster centers, each cluster center being defined as a multi-site co-evolution mode, with each co-evolution mode corresponding to a hyperedge type; for the current visit time point, the difference between the binary co-perceptual embedding vector of each graph node and the binary co-perceptual embedding vector of the previous visit time point is calculated to obtain the current difference vector; the current difference vector is matched with each cluster center to determine the co-evolution mode to which the current difference vector belongs; based on the identified co-evolution mode, a spatial hyperedge is constructed connecting at least three graph nodes participating in the co-evolution mode within the same visit time point; a temporal hyperedge is constructed connecting the nodes corresponding to the same graph node at multiple visit time points to model the temporal continuity of the visual state of the target region of the same body part.

[0076] In this embodiment, the construction operations of spatial hyperedges and temporal hyperedges in the cross-temporal hypergraph construction layer are divided into two stages: offline mining and online construction.

[0077] In the offline data mining phase, a training dataset of historical psoriasis patients was used to discover and define multi-site co-evolutionary patterns. Specifically, for each patient in the training dataset, pairs of data from each adjacent visit time point were obtained (e.g., ...). , The changes in the binary collaborative sensing embedding vectors of each graph node between (etc.) are analyzed. The differences in the binary collaborative sensing embedding vectors of each graph node between two adjacent visit time points t-1 and t are calculated for each part and concatenated to form a multi-part difference vector describing the changes in the collaborative state of all N body parts in that adjacent time point pair. ; in, Let represent the binary co-sensory embedding vector of the i-th body part at time point t. This difference vector as a whole describes the direction and magnitude of the change in the co-sensory state of all body parts within a time step. The difference vectors corresponding to all patients and all adjacent time points in the training dataset are summarized to obtain a set of difference vectors. Unsupervised clustering (e.g., K-means clustering or spectral clustering) is performed on this set of difference vectors. The number of clusters C is a preset hyperparameter, which can be set according to the data size and the desired pattern granularity (e.g., C=8 or C=16). After clustering, C cluster centers are obtained, denoted as . Each cluster center This represents a recurring pattern of multi-part coordinated state changes in historical data, which is defined as a multi-part co-evolutionary pattern. For example, one cluster center might represent a pattern of simultaneous improvement in several body parts, with some parts showing signs of stagnation, such as a decrease in trunk area index, a decrease in scalp area index, a decrease in upper limb area index, but a relatively stable lower leg area index; another cluster center might represent a pattern of stable area indices for most body parts, but a slight increase in the area indices for the palms and soles. Each co-evolutionary pattern... Corresponding to a superedge type This means that the hyperedge type is specifically used to connect graph nodes that exhibit the co-evolutionary pattern within a certain time step. Cluster center Representative feature vectors of this co-evolution pattern are saved for matching during the online construction phase. After the offline mining phase is completed, the cluster centers remain unchanged in the subsequent online inference phase.

[0078] During the online construction phase, for the current visit time t, the binary collaborative perception embedding vectors of each graph node at the current time point are first obtained. The binary collaborative sensing embedding vector of the previous visit time point t-1 (when For the first visit time point, hyperedges may not be constructed immediately, or a preset initialization strategy may be used. Calculate the difference between the two to obtain the current difference vector: ; Then, the current difference vector C cluster centers obtained in the offline phase Perform similarity comparisons one by one. Similarity metrics can include Euclidean distance, cosine similarity, or other suitable distance functions. Select the cluster center that is most similar to the current discrepancy vector (smallest distance or highest similarity). The co-evolution mode corresponding to this cluster center This refers to the co-evolution mode to which the current time step belongs.

[0079] After determining the co-evolution mode at the current time step Next, the spatial hyperedge construction operation is performed. Spatial Hyperedge This is used to connect the various graph nodes that jointly participate in this co-evolutionary pattern within the current visit time point t. Specifically, it analyzes the cluster centers. Based on the component values ​​in each dimension, determine the set of body parts that exhibit significant state changes in this co-evolution mode (e.g., the change magnitude exceeds a preset threshold, or the change direction is consistent with the main trend of the mode); connect at least three graph nodes corresponding to the body parts belonging to this set at the current time point through a spatial hyperedge. For example, if the cluster center If the components in the four dimensions of trunk, scalp, upper limbs, and lower legs exhibit a pattern of "significant decrease in trunk, significant decrease in scalp, significant decrease in upper limbs, and near zero in lower legs," then a spatial hyperedge can be constructed. This connects the graph node instances corresponding to these four body parts at the current visit time t. The physical significance of this spatial hyperedge is that it explicitly expresses that at the current time, the visual state of the skin lesions of these four body parts is in a macroscopic pattern of multi-part coordinated change, rather than changing in isolation.

[0080] Simultaneously, the construction operation for time-exceeding edges is performed. This is done for the graph nodes corresponding to each body part. Time exceeds the boundary This involves concatenating the node instances at multiple historical visit times. Specifically, a time hyperedge connects the node instances of the same graph node at the current visit time t. Node instance at the previous visit time point t-1 And node instances from earlier visit timestamps up to the first visit timetamp. node instances This forms a trans-temporal superedge that runs along the time axis: When there are only two historical visit points (i.e. When there are three or more historical visit times (i.e., ...), the time hyperedge connects the node instances corresponding to these two time points; when there are three or more historical visit times (i.e., ...), the time hyperedge connects the node instances corresponding to these two time points. When a temporal hyperedge connects node instances at three or more time points, it conforms to the basic definition of a hyperedge. The physical significance of a temporal hyperedge lies in the fact that it provides the model with a structured representation of the continuity and gradation of the visual state of the same body part target region across multiple time steps. That is, by connecting the same graph node instances distributed along the time axis with hyperedges, the model can directly aggregate the state information of the same part at different time points in subsequent hypergraph convolution operations, thereby more effectively capturing the temporal evolution pattern of the visual state of that part.

[0081] It should be noted that spatial hyperedges and temporal hyperedges can coexist and function independently. Spatial hyperedges are responsible for aggregating state information from multiple related locations within a single point in time to form a higher-order collaborative representation, while temporal hyperedges are responsible for connecting multiple time-point instances of the same location along the time axis to inject temporal continuity information. Together, they constitute the hyperedge set for the current visit time point. The set will be dynamically updated as the current visit time point is updated (when entering the next visit time point, new difference vectors are calculated, new co-evolution patterns are identified, and new hyperedges are constructed and added to the set).

[0082] Thus, through the two stages of offline mining and online construction, the cross-temporal hypergraph construction layer completed the construction of spatial and temporal hyperedges. Spatial hyperedges, by connecting at least three graph nodes participating in the same co-evolutionary pattern, overcome the limitation of ordinary graph models that can only express binary pairwise relationships, enabling explicit modeling of high-order collaborative patterns across multiple body parts. Temporal hyperedges, by connecting node instances of the same graph node at multiple visitation times, achieve a structured expression of the continuity and gradual change patterns of the visual state of the same body part over time. The set of hyperedges jointly constituted by both provides the co-evolutionary temporal predictor with rich inputs that contain both high-order spatial collaborative relationships and cross-temporal continuity information, significantly enhancing the model's ability to model complex multi-body co-evolutionary patterns.

[0083] Optionally, the co-evolutionary temporal predictor includes a hypergraph convolution enhancement module, a temporal coding module, and a forward prediction module; and step 4 specifically includes: in the hypergraph convolution enhancement module, for the multi-site co-state matrix at each visit time point, a hypergraph convolution operation is performed using the hyperedge set constructed by the spatial co-evolutionary hypergraph encoder; the hypergraph convolution operation updates the embedding representation of each graph node through attention aggregation from nodes to hyperedges and information distribution from hyperedges to nodes, to obtain the enhanced multi-site co-state matrix; in the temporal coding module, the enhanced sequence of multi-site co-state matrices at multiple visit time points is input into a recurrent neural network or a Transformer encoder to generate a latent vector representing the historical dynamic evolution of the co-relationship between target regions of each body part; in the forward prediction module, the latent vector is input into the forward prediction network to generate a sequence of predicted co-state matrices for several future visit time points.

[0084] In this embodiment, the co-evolutionary temporal predictor is the second core neural network model following the spatial co-evolutionary hypergraph encoder. Its function is to learn the dynamic evolution law of co-evolutionary relationships from the co-evolutionary state matrix sequence and hyperedge set of multiple historical visit times, and generate forward predictions for several future visit times accordingly. The predictor consists of three functional modules stacked sequentially: a hypergraph convolutional enhancement module, a temporal encoding module, and a forward prediction module. The specific operations of each module are as follows.

[0085] In the hypergraph convolution enhancement module, for the multi-site collaborative state matrix at each historical visit time point... The set of hyperedges constructed at the same time point using a spatial collaborative hypergraph encoder Hypergraph convolution operations are performed to inject high-order multi-part co-evolution information and cross-temporal continuity information carried by hyperedges into the embedding representations of each graph node. The hypergraph convolution operation consists of a two-stage attention computation process.

[0086] The first stage is attention aggregation from nodes to hyperedges. For each hyperedge in the hyperedge set... Suppose it is connected Each graph node (for spatial hyperedges, these nodes originate from multiple body parts participating in the same co-evolutionary mode at the same time point; for temporal hyperedges, these nodes originate from node instances of the same body part at multiple historical visit times). Extract the multi-body co-state matrix of each graph node connected by the hyperedge. Binary Cooperative Perception Embedded Vectors The attention pooling operation aggregates the embedding vectors of these nodes into a single hyperedge embedding vector. The attention pooling mechanism works as follows: A learnable attention network is used to calculate the contribution weight of each node within the hyperedge to the overall representation of the hyperedge. Then, based on these weights, the node embeddings are weighted and summed to obtain a hyperedge embedding vector that comprehensively represents the overall state of the multiple nodes covered by the hyperedge at the current time step. The dimension of the hyperedge embedding vector is consistent with the dimension D of the node embedding vector to facilitate subsequent information distribution calculations.

[0087] The second stage is the information distribution from the hyperedge to the node. For each graph node... Collect all hyperedges containing the node (including both spatial and temporal hyperedges connecting the node), and obtain the set of hyperedge embedding vectors calculated in the first stage. Using another learnable attention network, matching weights are calculated between the node and each hyperedge containing it, measuring the informational importance of each hyperedge for the node's current state update. Based on these matching weights, the embedding vectors of each hyperedge are weighted and summed to obtain the node's higher-order augmentation information. This augmentation information is then fused with the node's original binary cooperative perception embedding vector (e.g., through element-wise addition, linear projection after vector concatenation, or gated fusion) to obtain the hypergraph convolution-enhanced embedding vector of the graph node at the current visit time. The enhanced embedding vectors of all N graph nodes are arranged in rows to form the enhanced multi-part cooperative state matrix. .

[0088] The core value of hypergraph convolution enhancement lies in the following: by aggregating nodes to hyperedges, the individual state information scattered across multiple nodes is gathered into a hyperedge representation reflecting the overall state of higher-order co-evolutionary patterns; by distributing hyperedges to nodes, the overall state information of higher-order co-evolutionary patterns is fed back to each participating node, enabling each node's representation to retain its own binary cooperative perception information while further absorbing all the higher-order co-evolutionary patterns it participates in and the macroscopic context of cross-temporal continuity relationships. After this enhancement step, the embedding vector of each node not only "knows" which binary cooperative associations it has with other parts, but also "knows" the macroscopic evolutionary patterns of the multiple parts it is in and the evolutionary context of the same part on the time axis, thus providing a richer input representation for subsequent temporal modeling.

[0089] In the temporal coding module, the sequence of multi-site co-state matrices from multiple historical visit time points, enhanced by hypergraph convolution, is used. The data is input into a temporal modeling network in chronological order. This network can employ a recurrent neural network (such as a bidirectional gated recurrent unit (Bi-GRU) or a long short-term memory (LSTM) network) or a Transformer encoder based on a self-attention mechanism, among other sequence modeling architectures. Taking a bidirectional GRU as an example, for each time step t, the enhanced co-state matrix is... The GRU cells are flattened or input row by row. The GRU, through its internal update and reset mechanisms, selectively retains key evolutionary information from historical time steps while discarding less important details, updating its hidden state step by step. After sequential processing of all historical time steps, the temporal coding module finally outputs a latent vector representing the dynamic evolution of the cooperative relationships between target regions of various body parts throughout the entire historical observation period. ,in The dimension of the latent vector (e.g.) or The latent vector can be the hidden state of the last time step, a weighted average of the hidden states of all time steps, or the output vector corresponding to the CLS token in the Transformer encoder. It compresses all the temporal evolution information extracted from the historical multi-part coordinated state matrix sequence, which is a key information bottleneck connecting historical observation and future prediction.

[0090] In the forward prediction module, the latent vectors generated by the temporal coding module are... The input is fed into the feedforward prediction network to generate a sequence of predicted co-state matrices for multiple consecutive future visit times. The specific structure of the feedforward prediction network can be matched to the requirements of the prediction task: a multilayer perceptron can be used to directly output an L×N×D three-dimensional tensor at once, where L is the number of future visit times to be predicted (e.g., L=2, corresponding to week 24 and week 52 respectively). This tensor, after shape transformation, becomes the sequence of predicted co-state matrices for each future time point. Alternatively, a decoder structure based on recurrent neural networks can be used to extract latent vectors. Starting with the initial state, the predictive co-state matrix for each future time step is generated autoregressively. That is, at each step, the decoder uses the predicted output of the previous time step (or the true value from the training phase) as part of the input for the current time step, combining it with its hidden state to generate the predicted output for the current time step, until the prediction results for all L future time steps are generated. Each predictive co-state matrix... The format is completely consistent with the historical cooperative state matrix output in step 3, and each row of it is identical. The vector is embedded to represent the predicted state of the corresponding body part at the corresponding future time point. This vector will be restored by the visual indicator decoder in step 5 into a purely visual state indicator prediction value that can be intuitively understood.

[0091] Thus, through step 4 and the sequential processing of the three functional modules mentioned above, the co-evolutionary time-series predictor, starting from the multi-part co-state matrix sequence and hyperedge set at multiple historical visit time points, completes the entire time-series reasoning process from extracting evolutionary patterns from historical co-states to generating forward predictions of future co-states, outputting a predicted co-state matrix sequence. This provides core predictive data for the visual indicator decoding and evolutionary trajectory visualization in step 5.

[0092] The training process of the co-evolutionary time series predictor in this embodiment is described below.

[0093] The training of the co-evolutionary temporal predictor was completed offline, and joint end-to-end training was performed with the spatial co-evolutionary hypergraph encoder. The training data was the same as that used for the encoder training, consisting of a training dataset composed of skin images of psoriasis lesions from multiple time points and locations of a large number of psoriasis patients. For each patient in the training dataset, the visual state vector set at each time point was first obtained according to the processing flow in step 2, and then the multi-site co-state matrix and hyperedge set at each historical time point were obtained according to the processing flow in step 3.

[0094] During training, the co-state matrix sequence and hyperedge set of m+1 consecutive historical visit time points are used as input, and the model is required to predict the co-state matrix sequence of the next L future visit time points. In the training dataset, the true co-state matrix of each time point is known; that is, for historical time points, the true co-state matrix is ​​calculated by forward propagation in step 3. For the prediction target time points (these time points are also known historical records in the training dataset, but are classified as prediction targets), the corresponding true visual state vector set is obtained by processing the image data of the corresponding time points in step 2, and then the true co-state matrix is ​​calculated by forward propagation in step 3, which serves as the supervision signal for prediction training.

[0095] During the forward propagation phase, the co-evolutionary time series predictor receives the historical co-state matrix sequence and the set of hyperedges, and generates the predicted co-state matrix sequence according to the process of the three functional modules mentioned above. During the loss calculation phase, each predicted cooperative state matrix in the predicted sequence is... The actual co-state matrix at the corresponding time point Element-by-element comparisons are performed to calculate the prediction accuracy loss. In this embodiment, the prediction accuracy loss can be calculated using the mean squared error loss function: ; in, To predict the predicted value of the i-th body part in the d-th dimension of the cooperative state matrix, This corresponds to the actual value.

[0096] To further ensure the reasonableness and physical consistency of the prediction results, an auxiliary loss term can be introduced during training. This term is called smoothness loss. The variation magnitude of the cooperative state matrix between adjacent prediction time steps is constrained and calculated as follows: ; in, This represents the Frobenius norm of the matrix, which is the square root of the sum of the squares of all its elements. This loss term encourages a smooth and gradual evolutionary trajectory in the predictions, suppressing irrational jumps.

[0097] Loss of consistency in parts The predicted state of anatomically symmetrical body parts should not deviate too much: ; in, This is a pre-defined set of anatomically symmetrical body parts (e.g., {(left upper limb, right upper limb), (left lower leg, right lower leg), (left palmoplantar fossa, right palmoplantar fossa)}, etc.).

[0098] Model Fidelity Loss The predicted patterns of co-evolutionary state changes are encouraged to fall within the already explored space of reasonable co-evolutionary patterns: ; in, This is the difference vector of the embedding vectors of each node between adjacent prediction time steps. The cluster centers for each co-evolutionary pattern obtained during the training phase are represented by this term. This loss term calculates the squared distance between the predicted difference vector and its nearest co-evolutionary pattern cluster center. The smaller the distance, the more the predicted evolutionary pattern conforms to the patterns discovered from historical data.

[0099] The total loss function is a weighted sum of the losses mentioned above: ; in, , and The preset non-negative balance coefficient is used to adjust the contribution of each auxiliary loss to the main prediction loss (e.g., =0.1, =0.05, =0.1).

[0100] Optionally, step 5 further includes: obtaining the predicted visual state index values ​​of the target area of ​​each body part at several future visit time points, comparing them with the observed visual state index values ​​at the last visit time point among multiple visit time points, and calculating the direction of change of the visual state index of the target area of ​​each body part; comparing the direction of change of each body part with the direction of change of other body parts among multiple preset body parts, marking the body parts whose direction of change is inconsistent with the other body parts, and outputting a list of abnormal trend parts; enabling the Monte Carlo dropout method for the forward prediction network in the co-evolutionary time series predictor, performing multiple random forward propagations, and obtaining multiple predicted samples of the visual state index of the target area of ​​each body part at several future visit time points; based on the multiple predicted samples, calculating the prediction probability of the visual state index in different preset change ranges, generating and outputting the predicted probability distribution of the visual state index of the target area of ​​each body part.

[0101] In this embodiment of the application, in addition to generating the evolution trajectory prediction map of the target area of ​​each body part, step 5 can further perform two auxiliary analysis functions: trend abnormal part identification and prediction probability distribution output, so as to provide richer prediction information.

[0102] In the process of identifying abnormal trend areas, the first step is to obtain the predicted visual state indicators of the target area for each body part at several future visit time points. Specifically, for each body part... Take the visual indicator decoder output from step 5, at the time of the last predicted visit in the future. Area proportion index forecast value Simultaneously, the last historical visit time point among multiple historical visit time points for that body part was selected. Actual observed area ratio index (This value is calculated and recorded in step 2 during the historical data processing phase). Compare the two values ​​to calculate the change in the area percentage index of this body part from the last historical time point to the last predicted time point: ; Based on this change, the direction of change of visual state indicators for each body part's target area is determined. The direction of change can be categorized into three types: when... When the value is less than a preset threshold (e.g., -0.05), the direction of change is determined to be "decreasing"; when... When the value exceeds a preset threshold (e.g., +0.05), the direction of change is determined to be "upward"; when... When the value is between two thresholds, the direction of change is determined to be "flat". The above thresholds can be set according to actual application needs, and this application does not impose specific limitations on them.

[0103] Then, the directions of change for each body part are compared to identify areas with abnormal trends. Specifically, for each body part... The direction of its change is compared with all preset body parts except Compare the changes in each of the body parts other than the one mentioned above. The direction of change is inconsistent with the direction of change of at least one of the other body parts, for example, The direction of change is "rising" or "remaining level," while at least one other body part is "falling"; or If the direction of change for one body part is "descending," while at least one other body part changes in the direction of "ascending" or "remaining level," then that body part... Marked as areas of abnormal trend. All marked body parts are compiled into a list of areas of abnormal trend and presented as additional output information.

[0104] It is important to clarify that the aforementioned "abnormal trend areas" marking only indicates an objective visual phenomenon where the area ratio of the target region of that body part changes in a direction inconsistent with other body parts. For example, while the lesion area is shrinking in most body parts, the lesion area in a specific body part may show an expanding trend. This does not constitute any medical diagnosis or treatment recommendation. The purpose of this marking is to alert observers, from the perspective of multi-site collaborative analysis, those body parts that may deserve special attention and whose evolutionary behavior differs from other parts.

[0105] In the operation of predicting the probability distribution output, a randomness mechanism is introduced into the forward prediction network of the co-evolutionary time-series predictor to generate multiple prediction samples, thereby estimating the uncertainty of the prediction results. In this embodiment, Monte Carlo dropout is used to implement this operation. Dropout is a regularization technique commonly used in the training phase of neural networks. During forward propagation, it randomly sets the output of some neurons in the network to zero with a preset probability (e.g., 0.5) to enhance the generalization ability of the model. Monte Carlo dropout keeps dropout enabled during the inference phase, so that the combination of neurons randomly set to zero in the network is different each time forward propagation, thus producing slightly different prediction outputs. By performing such random forward propagation multiple times, multiple samples of the prediction results can be obtained.

[0106] In practice, the feedforward prediction network of the co-evolutionary temporal predictor in step 4 is set to training mode (i.e., maintaining the random dropout behavior of the dropout layer), and the same input data (i.e., the same latent vector) is used. The network performs M forward propagations (e.g., M=100). Because the combination of neurons randomly dropped in each propagation is different, the predicted co-state matrix sequence output by the network is not exactly the same. For each forward propagation, the predicted co-state matrix sequence is decoded by a visual index decoder into predicted visual state indices for each body part, ultimately yielding M independent predicted samples for each body part in each predicted index. For example, for a body part... Predicting visit timing The area proportion index is used to obtain the sample set. .

[0107] Based on the aforementioned M prediction samples, the predicted probability of the visual state index within different preset change ranges is calculated. These preset change ranges may include: "area proportion decreasing by 50% or more from the current level," "area proportion decreasing by 25% to 50% from the current level," "area proportion changing within ±25% of the current level" (i.e., remaining essentially unchanged), and "area proportion increasing by 25% or more from the current level." For each preset range, the number of samples in the M prediction samples that meet the conditions of that range is counted, and this number is divided by the total number of samples M to obtain an estimated probability that the predicted value falls within that range. For example: ; ; In the formula, This is an indicator function; it takes the value 1 when the condition within the parentheses is true, and 0 otherwise. Similarly, the predicted probability for various ranges of change can be calculated.

[0108] The predicted probabilities of each preset range of variation are presented in a visual format such as tables or probability density curves, generating and outputting the predicted probability distribution of visual state indicators for target areas of each body part. This probability distribution provides users with a quantitative reference for the confidence level of future prediction results; that is, the more concentrated the probability values ​​are within a specific range of variation, the higher the model's confidence in the prediction result; the more dispersed the probability values ​​are within each range, the greater the uncertainty of the prediction result.

[0109] Therefore, through the aforementioned two operations of identifying abnormal trend sites and outputting predicted probability distributions, step 5 not only provides a deterministic evolutionary trajectory prediction map of the visual state of target areas for each body part, but also additionally provides: on the one hand, an automatic analysis of the consistency of evolutionary behavior across different parts and its deviation markings, presented in the form of a list of abnormal trend sites; on the other hand, a quantitative estimate of the uncertainty of the prediction results, presented in the form of predicted probability distributions for each range of change. These two auxiliary outputs further enhance the practical value and information richness of this method in the application scenarios of long-term evolution monitoring and trend prediction of psoriatic lesions.

[0110] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs the method described in any of the above embodiments.

[0111] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0113] Example 2

[0114] Figure 4 A psoriatic lesion evolution trajectory prediction system based on multi-site temporal collaborative sensing, according to this embodiment, is illustrated. This system corresponds to the method described in Embodiment 1. (Reference) Figure 4 As shown, the system includes: an image acquisition module 410, used to acquire skin images of psoriasis lesions on multiple preset body parts of psoriasis patients at multiple visit time points; an image processing module 420, used to preprocess the skin images of psoriasis lesions on each body part at each visit time point, segment and extract target regions in the preprocessed skin images of psoriasis lesions, and calculate visual state representation vectors based on the target regions to obtain a set of visual state vectors for all preset body parts at each visit time point; the target regions are psoriasis lesion regions that differ from the visual features of the background skin; and an encoding module 430, used to input the set of visual state vectors for each visit time point into a pre-trained spatial collaborative hypergraph encoder to capture each... The binary collaborative association pattern between target regions of body parts outputs a multi-part collaborative state matrix and hyperedge set for each visit time point. The prediction module 440 is used to input the multi-part collaborative state matrix sequence and hyperedge set of all visit time points into a pre-trained collaborative evolution time series predictor. Based on the dynamic evolution law of the collaborative relationship between target regions of each body part, it generates a predicted collaborative state matrix sequence for several future visit time points. The decoding module 450 is used to decode the state embedding vector of each body part in the predicted collaborative state matrix sequence into the visual state index prediction value of the corresponding target region through a visual index decoder, and generate an evolution trajectory prediction map of the target region of each body part based on the visual state index prediction value.

[0115] It should be noted that the psoriasis lesion evolution trajectory prediction system based on multi-site temporal collaborative perception provided in this embodiment can realize all the functions and steps in the above method embodiments, solve the same technical problems, and achieve the same technical effects. The similarities will not be repeated.

[0116] Example 3

[0117] Figure 5 A psoriatic lesion evolution trajectory prediction system based on multi-site temporal collaborative sensing, according to this embodiment, is illustrated. This system corresponds to the method described in Embodiment 1. (Reference) Figure 5 As shown, the system includes: a processor 510; and a memory 520 connected to the processor 510, used to provide the processor 510 with instructions to process the following steps: acquiring skin images of psoriasis lesions on multiple preset body parts of a psoriasis patient at multiple visit time points; preprocessing the skin images of psoriasis lesions on each body part at each visit time point, segmenting and extracting target regions in the preprocessed skin images of psoriasis lesions, and calculating visual state representation vectors based on the target regions to obtain a set of visual state vectors for all preset body parts at each visit time point; the target region is the psoriasis lesion region that differs from the background skin visual features; and inputting the set of visual state vectors for each visit time point into the preprocessor. A pre-trained spatial cooperative hypergraph encoder captures the binary cooperative association patterns between target regions of each body part, outputting a multi-part cooperative state matrix and hyperedge set for each visit time point. The sequence of multi-part cooperative state matrices and hyperedge sets from all visit time points is input into a pre-trained cooperative evolution time-series predictor. Based on the dynamic evolution of the cooperative relationships between target regions of each body part, a predicted cooperative state matrix sequence for several future visit time points is generated. The state embedding vectors of each body part in the predicted cooperative state matrix sequence are decoded by a visual index decoder into predicted visual state index values ​​for the corresponding target regions. Based on these predicted visual state index values, an evolution trajectory prediction map of the target regions for each body part is generated.

[0118] It should be noted that the psoriasis lesion evolution trajectory prediction system based on multi-site temporal collaborative perception provided in this embodiment can realize all the functions and steps in the above method embodiments, solve the same technical problems, and achieve the same technical effects. The similarities will not be repeated.

[0119] It should be noted that the method and system for predicting the evolution trajectory of psoriasis lesions based on multi-site temporal collaborative perception provided in this application obtains predicted values, evolution trajectory curves, and predicted probability distributions of pure visual state indicators such as area proportion index, redness index, and scale texture index of target areas in various body parts through image acquisition, image preprocessing, target region segmentation, pure visual state vector extraction, spatial collaborative hypergraph coding, collaborative evolution temporal prediction, and visual indicator decoding and trajectory visualization. Essentially, it is an information processing method that processes and computer-aided analyzes the objective visual information contained in psoriasis lesion skin images. The predicted values ​​and evolution trajectory curves of the aforementioned pure visual state indicators only reflect the changing trends of the target area in terms of visual features such as color, area proportion, and texture over time. They are not directly used to determine the final diagnosis of psoriasis or the severity of the condition, but rather serve as intermediate or auxiliary reference information for professional physicians to use in conjunction with clinical symptoms, physical examination, laboratory tests, and other examination methods during clinical decision-making. Therefore, the method in this application does not constitute a disease diagnosis method in the sense of patent law.

[0120] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0121] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0126] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for predicting the evolution trajectory of psoriatic lesions based on multi-site temporal collaborative sensing, characterized in that, include: Step 1: Collect skin images of psoriatic lesions on multiple preset body parts at multiple visit time points for psoriasis patients; Step 2: Preprocess the psoriasis lesion skin images of each body part at each visit time point, segment and extract the target region in the preprocessed psoriasis lesion skin images, and calculate the visual state representation vector based on the target region to obtain the set of visual state vectors of all preset body parts at each visit time point; the target region is the psoriasis lesion region that differs from the background skin visual features. Step 3: Input the set of visual state vectors at each visit time point into the pre-trained spatial cooperative hypergraph encoder to capture the binary cooperative association patterns between target regions of each body part, and output the multi-part cooperative state matrix and hyperedge set at each visit time point. Step 4: Input the multi-site co-state matrix sequence and hyperedge set of all visit time points into the pre-trained co-evolution time series predictor, and generate the predicted co-state matrix sequence for several future visit time points based on the dynamic evolution law of the co-relationship between the target regions of each body part. Step 5: Decode the state embedding vector of each body part in the predicted co-state matrix sequence into the visual state index prediction value of the corresponding target region through the visual index decoder, and generate the evolution trajectory prediction map of the target region of each body part based on the visual state index prediction value.

2. The method according to claim 1, characterized in that, Step 2 specifically includes: By using a keypoint detection network, keypoint localization and affine transformation are performed on skin images of psoriatic lesions on each body part, aligning the images to standard pose and scale to obtain standardized images. A semantic segmentation network is used to perform pixel-by-pixel prediction on the standardized image, extracting target regions that differ from the visual features of the background skin, and generating a binary mask. Based on the pixel values ​​of the binary mask and the normalized image, a visual state representation vector of the target region for each body part is calculated. The visual state representation vector includes at least one of the following indices: area ratio index, which represents the proportion of the area of ​​the target region to the total area of ​​the normalized image of the body part; redness index, which represents the ratio of the intensity of the red channel to the intensity of the green and blue channels within the target region; and scale texture index, which represents the texture roughness statistics within the target region calculated based on the local binary pattern or gray-level co-occurrence matrix. The visual state representation vectors of all preset body parts at each visit time point are aggregated to form the visual state vector set of the visit time point.

3. The method according to claim 1, characterized in that, The spatial collaborative hypergraph encoder includes a heterogeneous node initialization layer, a multi-relationship composite adjacency message passing layer, and a cross-temporal hypergraph construction layer; Furthermore, step 3 specifically includes: In the heterogeneous node initialization layer, based on the set of visual state vectors at each visit time point, each body part is constructed as a graph node. The part type label is marked according to the histological characteristics of each body part. Using the independent learnable projection matrix corresponding to each part type, the visual state representation vector of each body part is mapped to the node initial embedding vector. The part type labels include: thick keratinized part, hair follicle-rich part, nail-like part, and regular skin part. In the multi-relationship composite adjacency message passing layer, multiple parallel relation subgraphs are constructed, each corresponding to a different binary association type between body parts. Based on the initial node embedding vector, message passing of the heterogeneous graph attention network is executed independently in each relation subgraph. The node update results of the multi-path relation subgraphs are fused through an adaptive gating mechanism to obtain the binary collaborative perception embedding vector of each body part, which constitutes the multi-part collaborative state matrix at the current visit time. In the cross-temporal hypergraph construction layer, based on the difference between the binary collaborative perception embedding vectors of each graph node at the current visit time point and the previous visit time point, and combined with the multi-site co-evolution patterns mined from historical psoriasis patient data, a spatial hyperedge connecting at least three graph nodes and a temporal hyperedge connecting the nodes corresponding to the same graph node at the multiple visit time points are constructed to form a hyperedge set for use by the co-evolutionary temporal predictor.

4. The method according to claim 3, characterized in that, In the multi-relation composite adjacency message passing layer, the operation of constructing multiple parallel relation subgraphs includes: Based on prior anatomical knowledge of the segments innervated by human nerves or the distribution areas of blood vessels, an anatomical adjacency subgraph is constructed; in the anatomical adjacency subgraph, the edges between nodes are determined to be pairs of body parts with anatomical associations based on the prior anatomical knowledge, and the weight of the edges is initialized by the strength of the anatomical association. Based on the correspondence between symmetrical parts of the body, a symmetrical adjacency subgraph is constructed; in the symmetrical adjacency subgraph, the edges between nodes are determined to be pairs of body parts that are symmetrically associated according to the correspondence, and the weight of the edges is initialized by the degree of synchronicity of symmetrical parts in the evolution of psoriatic lesions. Based on the asynchronous evolution pattern of skin lesions between different sites obtained from clinical data of psoriasis, a clinical co-evolution subgraph is constructed. In the clinical co-evolution subgraph, the edges between nodes are determined to be pairs of body sites with clinical synergy or asynchronous correlation according to the asynchronous evolution pattern, and the weights of the edges are obtained by pre-training the statistical correlation of the clinical data.

5. The method according to claim 3, characterized in that, The spatial hyperedge and the temporal hyperedge are constructed using the following operations: From historical psoriasis patient data, the differences in the binary collaborative perception embedding vectors of each graph node at adjacent visit time points are extracted to obtain multiple difference vectors; Unsupervised clustering is performed on the multiple difference vectors to obtain several cluster centers. Each cluster center is defined as a multi-part co-evolution mode, and each co-evolution mode corresponds to a hyperedge type. For the current visit time point, calculate the difference between the binary collaborative perception embedding vector of each graph node and the binary collaborative perception embedding vector of the previous visit time point to obtain the current difference vector; The current difference vector is matched with each cluster center to determine the co-evolution mode to which the current difference vector belongs; Based on the identified co-evolution pattern, construct a spatial hyperedge connecting at least three graph nodes that participate in the co-evolution pattern within the same visit time point; Construct temporal hyperedges connecting the nodes corresponding to the same graph node at each of the multiple visit time points to model the temporal continuity of the visual state of the target area of ​​the same body part.

6. The method according to claim 1, characterized in that, The co-evolutionary temporal predictor includes a hypergraph convolution enhancement module, a temporal coding module, and a forward prediction module; and step 4 specifically includes: In the hypergraph convolution enhancement module, for the multi-part co-state matrix at each visit time point, a hypergraph convolution operation is performed using the hyperedge set constructed by the spatial co-graph encoder; the hypergraph convolution operation updates the embedding representation of each graph node through attention aggregation from nodes to hyperedges and information distribution from hyperedges to nodes, thereby obtaining the enhanced multi-part co-state matrix. In the temporal coding module, the enhanced sequence of multi-site collaborative state matrices at multiple visit time points is input into a recurrent neural network or a Transformer encoder to generate a latent vector representing the historical dynamic evolution of the collaborative relationship between target regions of each body part. In the forward prediction module, the latent vector is input into the forward prediction network to generate a sequence of predicted co-state matrices for several future visit time points.

7. The method according to claim 1, characterized in that, Step 5 further includes: The predicted visual state index values ​​of the target areas of each body part are obtained at several future visit time points, and compared with the observed visual state index values ​​of the last visit time point among the multiple visit time points. The direction of change of the visual state index of the target areas of each body part is calculated. The direction of change of each body part is compared with the direction of change of the other body parts among the multiple preset body parts, and the body parts whose direction of change is inconsistent with the other body parts are marked, and a list of abnormal trend parts is output. The Monte Carlo dropout method is enabled in the forward prediction network of the co-evolutionary time-series predictor, and multiple random forward propagations are performed to obtain multiple predicted samples of the visual state index of the target area of ​​each body part at several future visit times. Based on the multiple prediction samples, the prediction probability of the visual state index is calculated within different preset ranges of change, and the prediction probability distribution of the visual state index of the target area of ​​each body part is generated and output.

8. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 7 is performed by a processor.

9. A system for predicting the evolution trajectory of psoriatic lesions based on multi-site temporal collaborative perception, characterized in that, include: The image acquisition module is used to acquire images of psoriasis lesions on multiple preset body parts of psoriasis patients at multiple visit time points; The image processing module is used to preprocess the psoriasis lesion skin images of each body part at each visit time point, segment and extract the target region in the preprocessed psoriasis lesion skin images, and calculate the visual state representation vector based on the target region to obtain the set of visual state vectors of all preset body parts at each visit time point. The target area is the psoriatic lesion area that differs from the visual features of the background skin; The encoding module is used to input the set of visual state vectors at each visit time point into a pre-trained spatial cooperative hypergraph encoder, capture the binary cooperative association patterns between target regions of each body part, and output the multi-part cooperative state matrix and hyperedge set at each visit time point. The prediction module is used to input the multi-site co-state matrix sequence and hyperedge set of all visit time points into the pre-trained co-evolution time series predictor, and generate the predicted co-state matrix sequence for several future visit time points based on the dynamic evolution law of the co-relationship between the target areas of each body part. The decoding module is used to decode the state embedding vector of each body part in the predicted co-state matrix sequence into the visual state index prediction value of the corresponding target area through the visual index decoder, and generate the evolution trajectory prediction map of the target area of ​​each body part based on the visual state index prediction value.

10. A system for predicting the evolution trajectory of psoriatic lesions based on multi-site temporal collaborative perception, characterized in that, include: processor; A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Collect skin images of psoriatic lesions on multiple pre-defined body parts at multiple visit time points of psoriasis patients; Preprocessing is performed on the psoriasis lesion skin images of each body part at each visit time point. The target region is segmented and extracted from the preprocessed psoriasis lesion skin images. The visual state representation vector is calculated based on the target region to obtain the set of visual state vectors of all preset body parts at each visit time point. The target region is the psoriasis lesion region that differs from the visual features of the background skin. The set of visual state vectors at each visit time point is input into a pre-trained spatial cooperative hypergraph encoder to capture the binary cooperative association patterns between target regions of each body part, and outputs the multi-part cooperative state matrix and hyperedge set at each visit time point. The multi-site co-state matrix sequence and hyperedge set of all visit time points are input into a pre-trained co-evolution time series predictor. Based on the dynamic evolution law of the co-relationship between target areas of each body part, a predicted co-state matrix sequence for several future visit time points is generated. The state embedding vectors of each body part in the predicted co-state matrix sequence are decoded into visual state index prediction values ​​for the corresponding target regions by a visual index decoder, and an evolution trajectory prediction map of the target regions for each body part is generated based on the visual state index prediction values.

Images