Stool consistency aided assessment system control method, system, device and storage medium

Abstract

The application discloses a stool state auxiliary evaluation system control method, system, device and storage medium, and relates to the technical field of data processing. The method extracts defecation features based on a stool image and extracts a text vector based on multi-dimensional medical records. Input features are constructed according to the defecation features and the text vector. The input features are input into a large language model, and an auxiliary evaluation result associated with a target evaluation object is determined through the large language model. The method deeply integrates stool visual features and clinical text information, and supplements clinical relevant background information through multi-dimensional medical records to solve the problem that a single image feature cannot reflect deep correlations such as causes and courses. The method deeply mines the internal correlation between the defecation visual features and the medical record text vector, and combines the reasoning ability of the large language model to reason out the auxiliary evaluation result, thereby effectively improving the accuracy of the auxiliary evaluation result generation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to control methods, systems, devices and storage media for a fecal state auxiliary assessment system. Background Technology

[0002] The stool condition assessment system can automatically assess stool condition based on visual characteristics such as shape and color, and output corresponding assessment results such as mild indigestion, providing auxiliary reference for home health monitoring.

[0003] Current mainstream stool condition assessment methods all rely on computer vision algorithms. They collect stool images and then extract visual features from the images to classify and determine the state of the stool, depending on a single visual data point to output the assessment results. However, these methods are easily affected by factors such as the shooting environment, lighting interference, and individual differences in the samples, which can lead to problems such as missing visual feature information and misjudgment of features, ultimately resulting in low accuracy of the output assessment results.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a control method, system, device, and storage medium for a stool state auxiliary assessment system, aiming to solve the technical problem of how to improve the accuracy of generating auxiliary assessment results based on stool image recognition.

[0006] To achieve the above objectives, this application proposes a control method for a stool state auxiliary assessment system, the method comprising: Defecation features are extracted from stool images, and text vectors are extracted from multidimensional medical records; Input features are constructed based on the defecation characteristics and the text vector; The input features are fed into a large language model, and the large language model is used to determine the auxiliary evaluation results associated with the target evaluation object.

[0007] In one embodiment, the step of extracting defecation features based on stool images includes: A stool image is acquired through an image acquisition module, and the stool image is preprocessed. After locating and segmenting the feces region from the feces image using a pre-trained feces image feature extraction model, the defecation features in the feces region are extracted.

[0008] In one embodiment, the step of locating and segmenting the feces region from the feces image using a pre-trained feces image feature extraction model includes: The encoder of the stool image feature extraction model extracts image features from the stool image; The decoder of the feces image feature extraction model upsamples the dimension of the image features to the pixel size of the feces image to obtain a semantic mask map of the feces image, wherein the semantic mask map is used to represent the probability value of each pixel in the feces image being a feces pixel; Based on the semantic mask image and the preset classification probability threshold, the feces region is located and segmented from the feces image.

[0009] In one embodiment, the defecation characteristics include one or more of the following: defecation frequency, defecation volume, shape characteristics, color characteristics, and texture characteristics; The trait characteristics include trait categories; The color features include one or more of the following: color category, color distribution ratio, HSV color information, color peak ratio, and color patch distribution uniformity. The texture features include one or more of the following: texture category, surface smoothness, texture entropy value, and texture gradient distribution.

[0010] In one embodiment, the preprocessing includes one or more of image cropping, white balance correction, illumination normalization, noise suppression, and resolution normalization.

[0011] In one embodiment, the image acquisition module includes one or more of the following: a built-in camera unit in a mobile terminal, a camera unit in a wearable device, an external camera module, and a camera unit in a medical testing device.

[0012] In one embodiment, the structure of the feces image feature extraction model includes one or more of the following: Convolutional Neural Networks; Visual Transformer; Convolutional-Transformer hybrid neural networks; Multi-branch deep neural networks based on attention mechanisms; Multi-scale feature pyramid network.

[0013] In one embodiment, the step of constructing input features based on the defecation features and the text vector includes: The text vector and the defecation features are encoded into a preset format to obtain standardized information; Prompt information is constructed based on the standardized information and the preset prompt word template; The prompt information is used as the input feature.

[0014] In one embodiment, the step of constructing input features based on the defecation features and the text vector includes: The defecation features are encoded into a vector format to obtain a defecation vector; The input features are obtained by concatenating the defecation vector with the text vector.

[0015] In one embodiment, the step of constructing input features based on the defecation features and the text vector further includes: The defecation features are encoded into a vector format to obtain a defecation vector; The defecation vector and the text vector are input into a multimodal Transformer, and the cross-attention weights between the defecation vector and the text vector are calculated through the cross-modal attention layer of the multimodal Transformer. Based on the cross-attention weights, the defecation vector and the text vector are weighted respectively; The weighted defecation vector is concatenated with the text vector to obtain the input features.

[0016] In one embodiment, the step of calculating the cross-attention weights between the defecation vector and the text vector through the cross-modal attention layer of the multimodal Transformer further includes: Using the defecation vector as the query vector and the text vector as the key and value vectors, calculate the first cross-attention weight of the defecation vector on the text vector; Using the text vector as the query vector and the defecation vector as the key and value vectors, calculate the second cross-attention weight of the text vector on the defecation vector.

[0017] In one embodiment, the step of constructing input features based on the defecation features and the text vector includes: Determine the mapping nodes of the text vector and the defecation features in a preset medical knowledge graph; In the preset medical knowledge graph, a preset number of associated nodes and associated edges connected to the mapping node are determined to obtain a sub-graph. The sub-graph is aggregated using a pre-trained graph neural network model, and the resulting node feature vectors are used as the input features.

[0018] In one embodiment, the multidimensional medical record includes one or more of the following: basic information, feeding information, accompanying symptoms, past medical history / medication history, examination information, physician's conclusions, and follow-up conclusions.

[0019] In one embodiment, the auxiliary assessment results include one or more of the following: risk warning level, digestion status type, candidate causes of suspicion corresponding to the digestion status type, and evidence points corresponding to the candidate causes of suspicion.

[0020] In one embodiment, the large language model includes a general large language model and a medical vertical large language model. In one embodiment, before the step of inputting the input features into a large language model to determine the auxiliary evaluation result associated with the target evaluation object, the method further includes: A set of instructions is constructed based on medical record information and bowel information from medical record samples; A response set is constructed based on the assessment labels in the medical record sample that correspond to the medical record information and the bowel information; The instruction set and the response set are input as instruction-response pairs into the base model, and the base model is trained to update the linear layer parameters of the base model, thereby obtaining the large language model.

[0021] In one embodiment, before the step of inputting the instruction set and the response set as instruction-response pairs into the base model and training the base model, the method further includes: The sum of the original weight matrix of each target linear layer in the base model and the preset low-rank increment matrix is ​​used as the weight matrix of the target linear layer.

[0022] In one embodiment, the original weight matrix includes a query projection matrix, a key projection matrix, and a value projection matrix in the attention module, and / or an up and down projection matrix in the feedforward neural network, wherein the rank of the low-rank increment matrix is ​​less than the dimension of the hidden layer feature vector of the pedestal model.

[0023] In one embodiment, the step of inputting the instruction set and the response set as instruction-response pairs into a base model, training the base model to update the linear layer parameters of the base model, and obtaining the large language model includes: The instruction-response pair is input into the base model to obtain the predicted evaluation label output by the base model, which is determined based on the weight matrix and the feature vector in the instruction set. Determine the lexical cross-entropy loss between the predicted evaluation label and the feature vector in the response set; The low-rank increment matrix is ​​updated based on the lexical cross-entropy loss to update the linear layer parameters of the base model, thereby obtaining the large language model.

[0024] In one embodiment, the step of updating the low-rank increment matrix based on the lexical cross-entropy loss to update the linear layer parameters of the base model and obtain the large language model includes: The gradient value of the lexical cross-entropy loss relative to the low-rank increment matrix is ​​determined using a preset backpropagation algorithm. Based on the low-rank increment matrix, the gradient value, and the hyperparameters of the base model, a new low-rank increment matrix is ​​determined; The sum of the new low-rank increment matrix and the weight matrix is ​​used as the new weight matrix of the target linear layer to update the linear layer parameters of the base model. The base model after updating the linear layer parameters is used as the large language model.

[0025] In one embodiment, the step of inputting the input features into a large language model to determine the auxiliary evaluation result associated with the target evaluation object includes: The input features are fed into the target linear layer in the large language model, the target linear layer including an attention module and a feedforward network; In each target linear layer, the feature vector corresponding to the linear layer is determined based on the output vector of the preceding linear layer and the weight matrix of the attention module, and the output vector of the linear layer is determined based on the feature vector and the upper and lower projection matrices of the feedforward network. The output vector of the last target linear layer is mapped to the probability value of the evaluation label through the output layer of the large language model; Based on the probability value, the auxiliary evaluation result corresponding to the input feature is determined.

[0026] Furthermore, to achieve the above objectives, this application also proposes a stool state auxiliary assessment system, which includes: The visual analysis module is used to extract defecation features based on stool images; The medical record parsing module is used to extract text vectors based on multidimensional medical record records; A multimodal fusion module is used to construct input features based on the defecation features and the text vector; A large language model is used to determine the auxiliary evaluation results associated with the target evaluation object based on the input features.

[0027] In addition, to achieve the above objectives, this application also proposes a control device for a defecation state auxiliary assessment system, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the defecation state auxiliary assessment system as described above.

[0028] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method of the defecation state auxiliary assessment system as described above.

[0029] This application provides a control method for a stool state auxiliary assessment system. It extracts defecation features from stool images and extracts text vectors from multidimensional medical records. Input features are constructed based on the defecation features and text vectors. These input features are then fed into a large language model, which determines the auxiliary assessment results associated with the target assessment object. This method overcomes the limitations of single-dimensional assessment by deeply integrating stool visual features with clinical text information. Multidimensional medical records supplement relevant clinical background information, addressing the problem that single image features cannot reflect deep connections such as etiology and disease course. By deeply mining the intrinsic correlation between defecation visual features and medical record text vectors, and combining this with the reasoning ability of the large language model, the auxiliary assessment results are inferred, effectively improving the accuracy of the auxiliary assessment results and providing reliable auxiliary support for clinical defecation state assessment. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating the control method of the stool state auxiliary assessment system of this application; Figure 2 A flowchart illustrating the control method of the stool state auxiliary assessment system of this application, as shown in Embodiment 2. Figure 3 A flowchart illustrating the control method of the stool state auxiliary assessment system of this application in Embodiment 3; Figure 4 This is a schematic diagram of the hardware operating environment of the control method of the fecal state auxiliary assessment system in the embodiments of this application.

[0033] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0034] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0035] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. It should be noted that all actions involving the acquisition of signals, information, or data in this application are performed in accordance with the relevant data protection laws and regulations of the country where the application is located, and with authorization from the owner of the corresponding device.

[0036] The stool condition assessment system can automatically assess stool condition based on visual characteristics such as shape and color, and output corresponding assessment results such as mild indigestion, providing auxiliary reference for home health monitoring.

[0037] Current mainstream stool condition assessment methods all rely on computer vision algorithms. They collect stool images and then extract visual features from the images to classify and determine the state of the stool, depending on a single visual data point to output the assessment results. However, these methods are easily affected by factors such as the shooting environment, lighting interference, and individual differences in the samples, which can lead to problems such as missing visual feature information and misjudgment of features, ultimately resulting in low accuracy of the output assessment results.

[0038] In view of the above problems, this application proposes a control method for a stool state auxiliary assessment system. This method extracts defecation features from stool images and extracts text vectors from multidimensional medical records. Input features are constructed based on the defecation features and text vectors. These input features are then fed into a large language model, which determines the auxiliary assessment results associated with the target assessment object. This method breaks through the limitations of single-dimensional assessment by deeply integrating stool visual features with clinical text information. Multidimensional medical records supplement relevant clinical background information, addressing the problem that single image features cannot reflect deep connections such as etiology and disease course. By deeply mining the intrinsic correlation between defecation visual features and medical record text vectors, and combining this with the reasoning ability of the large language model, the auxiliary assessment results are inferred, effectively improving the accuracy of the auxiliary assessment results and providing reliable auxiliary support for clinical defecation state assessment.

[0039] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a defecation status auxiliary assessment system. The following description uses a defecation status auxiliary assessment system as an example to illustrate this embodiment and the subsequent embodiments.

[0040] First Embodiment The first embodiment of this application provides a control method for a stool state auxiliary assessment system, referring to... Figure 1 In this embodiment, the control method of the defecation status auxiliary assessment system includes steps S10 to S30: Step S10: Extract defecation features based on stool images and extract text vectors based on multidimensional medical records.

[0041] It should be noted that defecation features are a set of structured visual features output by the defecation status auxiliary assessment system after automatic identification, segmentation and quantification of defecation images through a defecation image feature extraction model.

[0042] The aforementioned defecation characteristics include one or more of the following: frequency of defecation, amount of defecation, shape characteristics, color characteristics, and texture characteristics.

[0043] Stool characteristics are standardized classification results of stool form, including characteristic categories such as foamy stool, mucus stool, bloody stool, stool containing food residue, oily and shiny stool, stool containing undigested milk curds, watery stool, dry stool, and normal soft stool.

[0044] Color characteristics include one or more of the following: color category, color distribution ratio, HSV (Hue, Saturation, Value) color information, proportion of color peak, and uniformity of color distribution.

[0045] Texture features represent the surface texture information of feces extracted from feces images, including one or more of texture category, surface smoothness, texture entropy value, and texture gradient distribution.

[0046] Among them, the texture category is a standardized classification result of the surface texture pattern of the feces image, including smooth texture, rough texture, granular texture, flocculent texture, mucus thread texture, foamy porous texture, and paste-like uniform texture, etc.

[0047] Texture entropy is a quantitative indicator of texture complexity calculated based on the gray-level or color distribution of an image, used to reflect the uniformity of the texture on the surface of feces. As an alternative method for calculating texture entropy, the feces image can be converted into a single-channel grayscale image. The number of pixels at each gray level in the single-channel grayscale image can be counted, and the probability of occurrence p(i) of each pixel at each gray level can be calculated according to the formula... The texture entropy value is calculated, where E is the texture entropy value.

[0048] Texture gradient distribution is a quantitative feature describing the rate and direction of change of pixel brightness or color in a feces image. As an alternative method for calculating texture entropy, the Sobel gradient operator can be used to first calculate the horizontal gradient Gx and vertical gradient Gy of the feces image, and then the gradient magnitude and direction can be obtained using the following formula: θ = arctan2(Gy, Gx). Where, Let θ be the gradient magnitude and θ be the gradient direction. Based on the gradient magnitude and gradient direction, histograms are constructed to obtain the texture gradient distribution.

[0049] The aforementioned multidimensional medical record includes one or more of the following: basic information, feeding information, accompanying symptoms, past medical history / medication history, examination information, physician's conclusions, and follow-up conclusions.

[0050] For example, a multidimensional medical record may specifically include the following: Basic information: one or more of the following: age in months, sex, and weight; Feeding information: one or more of the following: breastfeeding, formula feeding, mixed breastfeeding and formula feeding, recent formula switching record, complementary food introduction, history of iron supplementation, history of probiotics, and daily water intake; Accompanying symptoms: fever, vomiting, abdominal distension, crying, dehydration, rash, appetite and mental status (one or more of these); Past medical history / medication history: one or more of the following: history of antibiotic use, vaccination, allergies, and premature birth; Examination information: one or more of the following: routine stool examination, fecal occult blood test, stool culture results, complete blood count, inflammatory markers, and abdominal ultrasound; Doctor's conclusion: One or more of the following: type of abnormal bowel movements, suspected cause, severity level, warning sign assessment, and medical diagnostic opinion; Follow-up outcomes include one or more of the following: symptom resolution, bowel movement recovery, improvement in physical signs, effectiveness of interventions, final diagnosis, and follow-up health status.

[0051] As an alternative solution for extracting defecation features based on stool images, stool images can be acquired through an image acquisition module, and the acquired stool images can be preprocessed. Then, a pre-trained stool image feature extraction model can be used to locate and segment the stool region from the preprocessed stool image, and defecation features in the stool region can be extracted.

[0052] The feces region refers to the effective pixel area occupied by the feces themselves in the feces image. Locating and segmenting the aforementioned feces region from the preprocessed feces image can eliminate interference from background, diapers, and lighting, thereby improving the accuracy of defecation feature recognition.

[0053] Optionally, the image acquisition module may include one or more of the following: a built-in camera unit of a mobile terminal such as a mobile phone or tablet, a camera unit of a wearable device, an external camera module, and a camera unit of a medical testing device.

[0054] Optionally, the above preprocessing of the stool image includes one or more of the following: image cropping, white balance correction, illumination normalization, noise suppression, and resolution normalization.

[0055] For example, after acquiring a stool image, the stool state-assisted assessment system first identifies target detection boxes in the stool image using a preset target detection algorithm, and then crops the stool image based on these boxes. Next, it calculates the RGB channel offset and color temperature deviation in the cropped stool image, and performs white balance correction on the R (red), G (green), and B (blue) channels based on these offsets and deviations. Then, it calculates the global brightness mean of the white balance-corrected stool image, adjusts the brightness of the stool image based on this mean, and finally smooths the image pixels using Gaussian filtering, median filtering, or bilateral filtering. Finally, it scales the smoothed stool image according to the target resolution of the stool image feature extraction model to standardize its resolution, completing the preprocessing of the stool image.

[0056] Furthermore, as a first option for locating and segmenting fecal regions from preprocessed fecal images using a pre-trained fecal image feature extraction model, image features in the fecal image can be extracted by the encoder of the fecal image feature extraction model; then, the dimensions of the image features can be upsampled to the pixel size of the fecal image by the decoder of the fecal image feature extraction model to obtain a semantic mask map of the fecal image, wherein the semantic mask map is used to represent the probability value of each pixel in the fecal image being a fecal pixel; then, based on the above semantic mask map and a preset classification probability threshold, the fecal region can be located and segmented from the fecal image.

[0057] For example, the fecal state-assisted assessment system inputs a preprocessed fecal image into the encoder of a fecal image feature extraction model. The encoder performs convolution, downsampling, and feature mapping on the image, progressively compressing its spatial dimensions and extracting information such as edges, colors, and textures from the fecal image. Specifically, the encoder takes the preprocessed fecal image as input, slides a pre-trained convolutional kernel across the fecal image row by row and column by column with a fixed stride, and performs a weighted summation operation between the convolutional kernel and the local pixel values ​​for each pixel within the sliding window to obtain and output a convolutional feature map. Next, the convolutional feature map is downsampled using max pooling, average pooling, or large stride convolution to obtain a downsampled feature map. Furthermore, the downsampled feature map is transformed in channel dimension through convolutional layers, and a non-linear activation function, such as ReLU (Rectified Linear Unit) or GELU (Gaussian Error Linear Unit), is used for feature mapping, ultimately outputting the encoder's image features. After obtaining the image features from the feces image, these features are input into the decoder of the feces image feature extraction model. The decoder performs dimensional upsampling on the image features, gradually restoring their spatial dimensions until they match the pixel dimensions of the original feces image, resulting in a high-resolution feature map with the same pixel dimensions as the feces image. Then, the upsampled high-resolution feature map is passed through a 1×1 convolutional layer of the decoder, performing pixel-by-pixel convolution operations to compress the multi-channel high-resolution feature map into a single-channel feature map. A sigmoid function is then applied to each pixel value in the single-channel feature map to calculate the probability value that the pixel is a feces pixel, resulting in the semantic mask image. The weights and biases of all convolutional kernels in the encoder and decoder are pre-trained. Finally, the probability values ​​of pixels in the semantic mask image are compared with a classification probability threshold. If the probability value of a pixel is greater than or equal to the classification probability threshold, the pixel is determined to be a feces pixel; if the probability value is less than the classification probability threshold, the pixel is determined to be a background pixel.

[0058] As a second alternative approach to locate and segment fecal regions from pre-processed fecal images using a pre-trained fecal image feature extraction model, instance segmentation can be employed to extract multiple independent fecal regions from the image. Specifically, the fecal state-assisted evaluation system inputs the pre-processed fecal image into the fecal image feature extraction model. The encoder of the model sequentially performs convolution, downsampling, and feature mapping on the fecal image to extract multi-scale feature maps. Based on preset feature thresholds such as texture clarity and color similarity, preliminary regions potentially containing fecal matter are selected from these multi-scale feature maps. For each selected preliminary region, rectangular bounding boxes are automatically drawn to generate several candidate region boxes. Each candidate region box corresponds to a potentially existing independent object or separated region in the image. Subsequently, region feature alignment and classification prediction are performed on the image features within each candidate region box. The confidence score of each candidate region box is calculated to determine whether the region corresponding to that candidate region box is a fecal region, and a pixel-level mask of the corresponding region is output simultaneously. Then, the candidate region boxes are filtered according to a preset confidence threshold, eliminating invalid regions with low confidence and retaining multiple fecal instance regions that meet the confidence requirements. Finally, all the retained fecal instance regions are integrated to obtain one or more independent fecal regions segmented from the fecal image, achieving accurate segmentation and localization of scattered multi-clump feces.

[0059] In this process, region feature alignment refers to mapping each candidate region box onto the multi-scale feature map output by the encoder. A sampling alignment method is then used to eliminate size discrepancies between the candidate region box and the multi-scale feature map. Specifically, based on the size of the multi-scale feature map, the candidate region box is uniformly divided into small grids corresponding to that size. Each small grid is sampled to obtain a standardized feature vector for each candidate region box, ensuring consistent feature dimensions for each candidate region. Classification prediction involves using a pre-defined classification layer to identify and judge the standardized feature vector corresponding to each candidate region box, and calculating the confidence level of the candidate region box.

[0060] Optionally, the structure of the above-mentioned fecal image feature extraction model may include a convolutional neural network, a visual Transformer, a convolutional-Transformer hybrid neural network, a multi-branch deep neural network based on an attention mechanism, or a multi-scale feature pyramid network.

[0061] Furthermore, as an alternative approach to extract defecation features from the fecal region, a multi-task learning method can be used to simultaneously extract multiple defecation features such as defecation frequency, defecation volume, shape features, color features, and texture features.

[0062] Step S20: Construct input features based on the defecation features and the text vector.

[0063] As a first option for constructing input features, defecation features can be encoded into a vector format to obtain a defecation vector, and the defecation vector can be concatenated with a text vector to obtain the input features.

[0064] For example, the above-mentioned defecation features are quantified and vectorized: for quantified features such as defecation frequency and volume, their values ​​are directly mapped to vector elements of the corresponding dimensions; for categorical or descriptive features such as morphological features, color features, and texture features, one-hot encoding or embedding encoding is used to convert them into vectors of fixed dimensions; finally, the vectors of all defecation features are integrated to obtain the defecation vector. Simultaneously, the text from the multidimensional medical record is converted into a fixed-dimensional text vector, and the defecation vector and the text vector obtained above are concatenated to obtain the above-mentioned input features.

[0065] As a second alternative for constructing input features, text vectors and defecation features are encoded into a preset format to obtain standardized information. Prompt information is then constructed based on the standardized information and prompt word templates, and the prompt information is used as input features.

[0066] For example, text vectors and defecation features are encoded into a preset format to obtain standardized information. This preset format can be a key-value pair format such as {"defecation volume":"medium", "color":"yellowish brown", "shape":"formed", "texture":"uniform"}, structured text format, JSON format, XML markup format, or fixed-dimensional vector format, etc. Next, a preset prompt word template is invoked. The prompt word template has a fixed sentence structure and reserves embedding positions for standardized information. The standardized information is embedded into the prompt word template one by one according to the reserved positions. Placeholders in the prompt word template are replaced to complete the splicing and construction of the prompt information, which is then used as input features.

[0067] As a third option for constructing input features, defecation features are encoded into vector format to obtain defecation vectors; the defecation vectors and text vectors are input into a multimodal Transformer, and the cross-attention weights between the defecation vectors and text vectors are calculated through the cross-modal attention layer of the multimodal Transformer; the defecation vectors and text vectors are weighted according to the cross-attention weights; the weighted defecation vectors and text vectors are concatenated to obtain the input features.

[0068] For example, the defecation features are first encoded into a vector format to obtain a defecation vector. This defecation vector and the text vector are then fed into a multimodal Transformer as input. The multimodal Transformer calls its own cross-modal attention layer to perform cross-attention calculation on the input defecation vector (modality 1) and text vector (modality 2). The cross-modal attention layer first performs linear projection on the defecation vector and text vector respectively to obtain their respective query vector (Q), key vector (K), and value vector (V). Then, it calculates the correlation between the defecation vector and text vector through an attention mechanism, outputting cross-attention weights that represent the degree of correlation between the two vectors. The weight values ​​of the cross-attention weights range from 0 to 1; the higher the weight value, the stronger the correlation between the corresponding vectors. Next, based on the above cross-attention weights, the defecation vector and text vector are weighted separately. The cross-attention weights are used to sum the weighted elements of each dimension of the defecation vector to obtain a weighted defecation vector. Simultaneously, the corresponding cross-attention weights are used to sum the weighted elements of each dimension of the text vector to obtain a weighted text vector. The core purpose of weighted processing is to strengthen the highly correlated features between the two vectors, suppress irrelevant features, and improve the accuracy of feature fusion. Finally, a vector concatenation operation is performed on the weighted defecation vector and the weighted text vector to obtain the input features.

[0069] Optionally, the steps described above for calculating the cross-attention weights between the defecation vector and the text vector using the cross-modal attention layer of the multimodal Transformer may further include: using the defecation vector as the query vector and the text vector as the key and value vectors, calculating the first cross-attention weights of the defecation vector on the text vector; and using the text vector as the query vector and the defecation vector as the key and value vectors, calculating the second cross-attention weights of the text vector on the defecation vector.

[0070] For example, suppose the defecation vector is a tensor of shape (S_b, d_model), where S_b is the length of the defecation sequence and d_model is the dimension of the hidden layer of the model. The text vector is a tensor of shape (S_t, d_model), where S_t is the length of the text sequence.

[0071] The steps for calculating the first cross-attention weights of the defecation vector and the text vector include: First, processing the defecation vector and text vector respectively through three learnable linear transformation layers to generate three core tensors for attention calculation. Specifically, the defecation vector is used as input and projected through the linear layer W_q1 to obtain the query vector Q_b with shape (S_b, d_k); the text vector is used as input and projected through the linear layer W_k1 to obtain the key vector K_t with shape (S_t, d_t); the text vector is used as input and projected through the linear layer W_v1 to obtain the value vector V_t with shape (S_t, d_v), where d_v is the dimension of the value vector. Then, matrix multiplication is performed on the transposes of Q_b and K_t obtained in the previous step. Specifically, the dot product of Q_b and K_t^T is calculated to obtain a fractional matrix with shape (S_b, S_t). Each element of this fractional matrix quantifies the correlation strength between a position in the defecation sequence and all positions in the text sequence. The score matrix obtained in step 2 is scaled by dividing it by sqrt(d_k), where d_k is the dimension of the key vector and query vector, which is less than or equal to the hidden layer dimension d_model of the model. Finally, the scaled score matrix is ​​normalized row-wise using the Softmax function. This ensures that for each query position of the "defecation vector," the sum of the attention weights for all key positions of the "text vectors" is 1. The final output of this step is the first cross-attention weight matrix, with shape (S_b, S_t).

[0072] The steps for calculating the second cross-attention weight of the text vector to the defecation vector are similar to those for calculating the first cross-attention weight, and will not be repeated here.

[0073] As a fourth option for constructing input features, text vectors and defecation features are determined and mapped to nodes in a preset medical knowledge graph; associated nodes and edges connected to the mapped nodes are determined in the preset medical knowledge graph for a preset number of steps to obtain a sub-graph; the sub-graph is aggregated through a pre-trained graph neural network model, and the node feature vectors obtained from the aggregation are used as input features.

[0074] The pre-built medical knowledge graph is a graph-structured knowledge base stored in the form of "entity-relationship-entity" triples, where both entities and relations have unique identifiers. For example, text vectors and defecation features are used as input. Based on semantic similarity calculation or pre-defined mapping rules, searches and matches are performed in the entity set of the pre-built medical knowledge graph to determine the mapping nodes corresponding to the text vectors, denoted as node set N_text, and the mapping nodes corresponding to the defecation features, denoted as node set N_feat. These two sets are merged to obtain the initial mapping node set N_map = N_text ∪ N_feat. Starting from each mapping node in this mapping node set N_map, a breadth-first traversal or random walk is performed along the associated edges in the pre-built medical knowledge graph to collect all neighboring nodes within a pre-defined number of hops and the edges connecting these neighboring nodes, resulting in a sub-graph. The sub-graph contains the original mapping node and its associated nodes, i.e., other entity nodes connected to the mapping node collected within the pre-defined number of hops, as well as the relationship edges connecting all the aforementioned mapping nodes and associated nodes. Let V_sub denote the set of nodes in the subgraph and E_sub denote the set of edges. Next, assign an initial feature vector to each node in the V_sub subgraph. Optionally, if the medical knowledge graph itself has entity features such as description vectors or type encodings, then directly use the entity features as the initial feature vector. Otherwise, assign a learnable embedding vector or its one-hot encoded projection to each node to obtain the initial feature matrix X for all nodes, with the shape (|V_sub|, d), where |V_sub| is the total number of nodes in the subgraph, and d is the node feature dimension. Finally, input the subgraph and the initial node feature matrix X into a pre-trained graph neural network model. The graph neural network model performs multi-layer message passing: at each layer, each node aggregates the feature information of its direct neighbors and, combined with its own features, updates its feature representation through a nonlinear transformation to obtain the updated feature matrix H for all nodes, still with the shape (|V_sub|, d'), where d' is the node feature dimension after aggregation at each layer. Each row in matrix H, i.e., the feature vector of each node, incorporates its own context information and that of the multi-hop associated edge structure. The updated node feature matrix H obtained above is then aggregated, compressing the variable-dimensional node feature matrix into a fixed-dimensional global graph representation vector, which serves as the input feature.

[0075] Step S30: Input the input features into the large language model, and determine the auxiliary evaluation result associated with the target evaluation object through the large language model.

[0076] It should be noted that large language models include general-purpose large language models such as GPT, Gemini, Doubao, Wenxin Yiyan, and Qianwen, as well as medical vertical large language models such as Xunfei Xinghuo Medical, Jingyi Qianxun, Qianwen Health, and OpenMEDLab Pudong Medical.

[0077] For example, the aforementioned input features are fed into a large language model. The large language model then performs reasoning based on its learned medical knowledge to generate auxiliary assessment results to support clinical decision-making. These auxiliary assessment results include one or more of the following: risk warning level, digestive state type, candidate causes of suspicion corresponding to the digestive state type, and evidence points corresponding to the candidate causes of suspicion.

[0078] For example, the digestive state type can be diarrhea, constipation, bloody stool, mucus stool, steatorrhea tendency, or normal variation; the severity level can be expressed numerically, such as 0 = observable, 1 = outpatient visit recommended, 2 = emergency recommended; the suspected causes can be infectious, lactose intolerance, secondary, food protein allergy, feeding-related, drug-related, and functional gastrointestinal disorders; the key evidence can be the key triggering factors extracted from the input, such as age, bloody stool, dehydration, fever, and changing formula.

[0079] This embodiment first uses stool images as a basis, employing a pre-processed and pre-trained image feature extraction model to locate and segment the stool region, extracting structured defecation features such as defecation frequency and characteristics. Simultaneously, text vectors are extracted from multi-dimensional medical records. These two methods are then fused to construct input features, which are then fed into a large language model. The large language model outputs auxiliary assessment results. This method overcomes the limitations of single-dimensional assessment by deeply integrating stool visual features with clinical text information. Multi-dimensional medical records supplement relevant clinical background information, addressing the problem that single image features cannot reflect deep connections such as etiology and disease progression. By deeply mining the intrinsic relationship between defecation visual features and medical record text vectors, and combining this with the reasoning ability of the large language model, auxiliary assessment results are derived, effectively improving the accuracy of the auxiliary assessment results and providing reliable support for clinical defecation status assessment.

[0080] Second Embodiment Based on the first embodiment described above, in the control method of the stool state auxiliary assessment system proposed in this embodiment, referring to... Figure 2 Before step S10, steps S40 to S60 are also included: Step S40: Construct an instruction set based on medical record information and bowel movement information in the medical record sample.

[0081] A large number of medical record samples are acquired, and each sample is iterated through to extract the medical record information text field and the defecation information text field. The medical record information includes one or more of the following: basic information, feeding information, accompanying symptoms, past medical history / medication history, and examination information. These medical record information and defecation information text fields are then populated and combined according to a pre-defined instruction template to construct an instruction set. This instruction template is used to transform the raw data into a natural language instruction that guides the model to perform a specific task. For example, the instruction template could be: "Please analyze based on the following medical record information and defecation details: [Enter medical record information here]. Defecation record: [Enter defecation information here]. Please provide the corresponding assessment." The instruction set contains the same number of instruction texts as the number of medical record samples. Each text is a structured natural language instruction containing specific sample data, used as subsequent input to the base model.

[0082] Step S50: Construct a response set based on the evaluation tags in the medical record sample that correspond to the medical record information and the defecation information.

[0083] The aforementioned assessment labels include physician conclusions and / or follow-up conclusions corresponding to the medical record information and bowel information in the medical record samples. The "assessment labels" for each medical record sample are organized according to a preset response format to obtain response text, such as: "Assessment Result: [Enter assessment label here].", to ensure that each response and the corresponding instruction constructed in step S40 form a correct question-and-answer pair in content.

[0084] Step S60: Input the instruction set and the response set as instruction-response pairs into the base model, train the base model to update the linear layer parameters of the base model, and obtain the large language model.

[0085] Optionally, before training begins, the parameters in the Transformer block of the base model can be frozen, i.e., set to be untrainable, to preserve the knowledge and language capabilities acquired during model pre-training. Simultaneously, it can be specified that only the parameters of the preset linear layers at the top of the model, the output projection layers, or the adapter layers added for fine-tuning are unfrozen, allowing them to be trained and updated.

[0086] For example, the above instruction set and response set are concatenated to form a complete "instruction-response pair" text sequence, which serves as training data. The instruction text from the training data is input into the base model, which performs autoregressive generation, calculates and outputs the predicted probability distribution for each word in the response text. Then, using the cross-entropy loss function, the predicted probability distribution for each word in the response text output by the base model is compared with the actual response text words to calculate the loss value. Based on this loss value, the parameters of the unfrozen linear layer or the newly added adapter layer are updated via backpropagation to obtain a large language model for the defecation state auxiliary assessment task.

[0087] This embodiment introduces a parameter-efficient instruction fine-tuning method based on the base large language model to construct a domain model specifically for stool state auxiliary assessment. The method first systematically extracts structured medical record information, defecation information, and corresponding doctor's conclusions or follow-up conclusions from a massive medical record sample, and constructs them into standardized instruction and response sets using preset templates. Subsequently, during training, the parameters of the base model's Transformer backbone are frozen to fully preserve its pre-trained general knowledge and language understanding capabilities, updating only the parameters of the top linear layer or adapter. This allows the base model to learn reliable medical reasoning and professional judgment with lower parameter tuning costs, improving the accuracy of the assessment.

[0088] Third Embodiment Based on the first and second embodiments described above, in the control method of the defecation state auxiliary assessment system proposed in this embodiment, referring to... Figure 3 Before step S60 above, step S70 may also be included: Step S70: The sum of the original weight matrix of each target linear layer in the base model and the preset low-rank increment matrix is ​​used as the weight matrix of the target linear layer.

[0089] The target linear layer is a linear transformation layer specified in the base model for training and fine-tuning. Preferably, the original weight matrix of the target linear layer may include the "query projection matrix" (Wq), "key projection matrix" (Wk), and "value projection matrix" (Wv) in the attention module of the base model; and the "up projection matrix" (Wup) and "down projection matrix" (Wdown) in the feedforward neural network.

[0090] For example, based on predefined layer names or structural rules, the parameter matrix of the target linear layer is determined by traversing the parameter space of the model, resulting in a list of original weight matrices W_original containing all "target linear layers". Each original weight matrix W_original has the shape (d_in, d_out), where d_in is the input feature dimension and d_out is the output feature dimension.

[0091] Then, a low-rank increment matrix is ​​initialized for each target linear layer. Specifically, for each original weight matrix W_original, two smaller matrices are initialized first: Matrix A has a shape of (d_in, r); it can be initialized with all zeros or with Gaussian random initialization.

[0092] Matrix B has a shape of (r, d_out) and is initialized with all zeros.

[0093] The rank r of matrices A and B is preset and satisfies that r is less than min(d_in, d_out).

[0094] The product of matrices A and B, A×B, constitutes the aforementioned low-rank increment matrix ΔW. Theoretically, the complete shape of the low-rank increment matrix is ​​the same as W_original, being (d_in, d_out). However, since it is a low-rank decomposition, the actual parameters stored and updated are only matrices A and B, and the rank of the low-rank increment matrix is ​​less than the dimension of the hidden layer feature vectors of the base model. Through the above steps, for each target linear layer, a pair of trainable low-rank matrices A and B are obtained, and the parameters of the original weight matrix W_original are frozen.

[0095] After determining the aforementioned low-rank increment matrix, the forward propagation function of the target linear layer is modified. When the input feature vector x is passed to the target linear layer, instead of simply calculating x × W_original, x × (W_original + ΔW) is calculated, where the low-rank increment matrix ΔW = A × B. The weight matrix of the target linear layer is reconstructed as the sum of the original weight matrix and the preset low-rank increment matrix, i.e., the weight matrix W_new = W_original + A × B. However, in terms of physical storage, the original weight matrix W_original remains unchanged and is not trainable; only a small number of matrices A and B are trainable parameters.

[0096] Furthermore, in this embodiment, step S60 includes steps S61 to S63: Step S61: Input the instruction-response pair into the base model to obtain the predicted evaluation label output by the base model based on the weight matrix and the feature vector in the instruction set.

[0097] The instruction and response texts in each instruction-response pair are concatenated into a complete text sequence, separated by a specific delimiter. This complete text sequence is then converted into a series of discrete token identifiers using the pedigree model's vocabulary. Simultaneously, an attention mask is generated to distinguish the real text from the padding, and a label mask is generated to mark the token positions of the response portion only for subsequent loss calculation.

[0098] Next, the aforementioned sequence of lexical identifiers is input into the base model. The embedding layer of the base model maps the sequence of lexical identifiers into a dense sequence of word embedding vectors. This sequence of word embedding vectors then flows into multiple Transformer layers for processing. Within each Transformer layer, the target linear layer performs a linear transformation using its weight matrix W_new = W_original + A × B. That is, the input vector simultaneously passes through the frozen original weight matrix and a trainable low-rank increment matrix path, and the results are summed before output, thus achieving task adaptation without updating most of the original parameters. After multiple transformations, the final hidden state is mapped to the vocabulary space through a trainable output projection layer. Finally, based on the label mask, the predicted probability distribution corresponding to the response part is extracted from the complete logarithmic probability distribution output by the base model. This predicted probability distribution serves as the prediction evaluation label, indicating the probability that each lexical in the vocabulary will become the next word when predicting each lexical in the response text, based on the generated context.

[0099] Step S62: Determine the lexical cross-entropy loss between the predicted evaluation label and the feature vector in the response set.

[0100] For example, the response text is also converted into a sequence of token identifiers, which serves as the ground truth label for training. Each token identifier in this sequence represents the ground truth token that the base model should predict at the corresponding position. For each token position in the token identifier sequence corresponding to the response text, the token cross-entropy loss between the predicted probability distribution of the base model and the ground truth token (one-hot encoding) is calculated. Specifically, the logarithm of the predicted probability distribution is multiplied by the one-hot encoding of the ground truth token and then summed. Finally, the token cross-entropy loss is averaged for all token identifier sequence positions corresponding to all response texts and for all instruction-response pairs within the same batch.

[0101] Step S63: Update the low-rank increment matrix according to the lexical cross-entropy loss to update the linear layer parameters of the base model, thereby obtaining the large language model.

[0102] Specifically, step S63 above also includes steps S631 to S634: Step S631: Determine the gradient value of the lexical cross-entropy loss relative to the low-rank increment matrix using a preset backpropagation algorithm.

[0103] For example, a pre-defined backpropagation algorithm is used, starting with the word cross-entropy loss. The algorithm then traverses backwards, dynamically constructing a complete computation graph during the forward propagation. This computation graph contains each operational node that participates in the forward propagation and generates outputs, such as matrix multiplication, addition, activation functions, softmax, and cross-entropy loss functions. Starting from the loss node (i.e., the cross-entropy loss function), each operational node is visited sequentially in reverse order, receiving the downstream gradient from its output. For the loss node itself, the downstream gradient is initially set to 1. Based on the Jacobian matrix of the operational node, the local gradient of the operational node with respect to each input variable is calculated. The downstream gradient is multiplied by the local gradient to obtain the gradient of the operational node with respect to each input variable, and this gradient is then passed to the upstream node corresponding to that input variable. This gradient propagation process continues until the leaf nodes representing the low-rank increment matrix parameters (i.e., matrices A and B) in the computation graph are reached. Based on the received gradients, the partial derivatives of the loss value with respect to each scalar element in these two matrices are calculated to obtain the gradient value of the low-rank increment matrix.

[0104] Step S632: Determine a new low-rank increment matrix based on the low-rank increment matrix, the gradient value, and the hyperparameters of the base model.

[0105] The pre-defined optimizer iteratively calculates new scalar elements based on its update rules and pre-defined hyperparameters such as the learning rate, combined with the current scalar elements of the low-rank increment matrix and their gradient values. For example, the update rule followed by stochastic gradient descent is: new scalar element = current scalar element - learning rate × gradient. More commonly used optimizers, such as Adam (Adaptive Moment Estimation), combine the first and second moment estimates of the gradient for a more complex adaptive update. This process is performed in each training iteration, aiming to reduce the word cross-entropy loss, thereby driving the parameters of the low-rank increment matrix towards a better direction.

[0106] Step S633: The sum of the new low-rank increment matrix and the weight matrix is ​​used as the new weight matrix of the target linear layer to update the linear layer parameters of the base model.

[0107] Step S634: The base model after updating the linear layer parameters is used as the large language model.

[0108] This embodiment employs a parameter-efficient fine-tuning method, using the backpropagation algorithm to calculate the gradient value of the word cross-entropy loss relative to the low-rank increment matrix. Then, the optimizer iteratively calculates new low-rank increment matrix parameters based on this gradient value, the current scalar elements of the low-rank increment matrix, and hyperparameters such as the learning rate, following a specific update rule. Finally, during forward propagation of the base model, the new low-rank increment matrix is ​​added to the frozen original weight matrix, serving as the effective weights for the target linear layer. This functionally updates the model parameters, achieving high efficiency in training and storage by updating only the very low-rank parameters.

[0109] Fourth embodiment Based on the above embodiments, in the control method of the convenient state-assisted assessment system provided in this embodiment, step S30 includes steps S31 to S34: Step S31: Input the input features into the target linear layer in the large language model. The target linear layer includes an attention module and a feedforward network.

[0110] Step S32: In each target linear layer, the feature vector corresponding to the linear layer is determined according to the output vector of the preceding linear layer and the weight matrix of the attention module, and the output vector of the linear layer is determined according to the feature vector and the upper and lower projection matrices of the feedforward network.

[0111] For example, the large language model receives input features and uses them as initial feature vectors, feeding them into the first target linear layer within the large language model as input data for this first target linear layer. In each target linear layer, the attention module receives the feature vector output from the preceding linear layer, denoted as the preceding output vector. According to the pre-trained parameters within the model, the preceding output vector is linearly transformed with the query projection matrix, key projection matrix, and value projection matrix respectively to calculate the attention weights between features. Then, the preceding output vector is weighted and aggregated with the aforementioned attention weights to generate the feature vector of the current target linear layer after attention processing. The feedforward network receives the feature vector output from the attention module; it first performs a linear transformation with the upper projection matrix to increase the feature dimension, then processes it with a non-linear activation function, and finally performs a linear transformation with the lower projection matrix to reduce the dimension to the target dimension, completing the feature mapping and generating the output vector of the current target linear layer. Then, the output vector of the current target linear layer is used as the preceding linear layer output vector of the next target linear layer, and the above steps continue until the last target linear layer is reached.

[0112] Step S33: The output vector of the last target linear layer is mapped to the probability value of the evaluation label through the output layer of the large language model.

[0113] Step S34: Determine the auxiliary evaluation result corresponding to the input feature based on the probability value.

[0114] For example, the output layer receives the output vector of the last target linear layer, performs a linear transformation on the output vector and the output layer weight matrix, and maps it to the dimension space corresponding to the evaluation label to obtain the predicted probability value of each evaluation label. Then, the large language model selects the evaluation label with the highest probability value as the core judgment result according to the preset decision rules, and organizes the evaluation label into a structured auxiliary evaluation result.

[0115] The above embodiments can make more accurate predictions by utilizing the pre-trained large language model, deeply integrate the visual features of stool with multidimensional medical record text, eliminate misjudgments caused by environmental interference and individual differences, and significantly improve the accuracy of assessment.

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control method of the fecal state auxiliary assessment system of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0117] This application provides a control device for a defecation status auxiliary assessment system. The control device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the defecation status auxiliary assessment system in the above embodiment 1.

[0118] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a control device suitable for implementing the defecation status auxiliary assessment system of the embodiments of this application. The control device for the defecation status auxiliary assessment system in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, and tablet computers (PADs), as well as fixed terminals such as desktop computers. Figure 4 The control device of the stool state auxiliary assessment system shown is merely an example and should not impose any limitation on the function and scope of use of the embodiments of this application.

[0119] like Figure 4As shown, the control device of the defecation status auxiliary assessment system may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the control device of the defecation status auxiliary assessment system. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the control device of the defecation status assistance assessment system to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a control device for a defecation status assistance assessment system with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0120] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0121] The control device for the stool state auxiliary assessment system provided in this application, employing the control method of the stool state auxiliary assessment system in the above embodiments, can solve the technical problem of how to improve the accuracy of generating auxiliary assessment results based on stool image recognition. Compared with the prior art, the beneficial effects of the control device for the stool state auxiliary assessment system provided in this application are the same as the beneficial effects of the control method for the stool state auxiliary assessment system provided in the above embodiments, and other technical features in the control device for the stool state auxiliary assessment system are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0122] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0124] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control method of the defecation state auxiliary assessment system in the above embodiments.

[0125] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0126] The aforementioned computer-readable storage medium may be included in the control device of the defecation status auxiliary assessment system; or it may exist independently and not be assembled into the control device of the defecation status auxiliary assessment system.

[0127] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the control device of the defecation status assistance assessment system, enable the control device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, or as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0130] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the control method of the above-described stool state auxiliary assessment system, which can solve the technical problem of how to improve the generation accuracy of auxiliary assessment results based on stool image recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the control method of the stool state auxiliary assessment system provided in the above embodiments, and will not be repeated here.

[0131] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the control method for the defecation state auxiliary assessment system described above.

[0132] The computer program product provided in this application solves the technical problem of how to improve the accuracy of generating auxiliary assessment results based on stool image recognition. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the control method of the stool state auxiliary assessment system provided in the above embodiments, and will not be repeated here.

[0133] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method for a stool state auxiliary assessment system, characterized in that, The control method of the stool condition auxiliary assessment system includes the following steps: Defecation features are extracted from stool images, and text vectors are extracted from multidimensional medical records; Input features are constructed based on the defecation characteristics and the text vector; The input features are fed into a large language model, and the large language model is used to determine the auxiliary evaluation results associated with the target evaluation object.

2. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The steps for extracting defecation features based on stool images include: A stool image is acquired through an image acquisition module, and the stool image is preprocessed. After locating and segmenting the feces region from the feces image using a pre-trained feces image feature extraction model, the defecation features in the feces region are extracted.

3. The control method of the stool state auxiliary assessment system as described in claim 2, characterized in that, The step of locating and segmenting the feces region from the feces image using the pre-trained feces image feature extraction model includes: The encoder of the stool image feature extraction model extracts image features from the stool image; The decoder of the feces image feature extraction model upsamples the dimension of the image features to the pixel size of the feces image to obtain a semantic mask map of the feces image, wherein the semantic mask map is used to represent the probability value of each pixel in the feces image being a feces pixel; Based on the semantic mask image and the preset classification probability threshold, the feces region is located and segmented from the feces image.

4. The control method of the stool state auxiliary assessment system as described in claim 2, characterized in that, The defecation characteristics include one or more of the following: defecation frequency, defecation volume, stool shape, color, and texture. The trait characteristics include trait categories; The color features include one or more of the following: color category, color distribution ratio, HSV color information, color peak ratio, and color patch distribution uniformity. The texture features include one or more of the following: texture category, surface smoothness, texture entropy value, and texture gradient distribution.

5. The control method of the stool state auxiliary assessment system as described in claim 2, characterized in that, The preprocessing includes one or more of the following: image cropping, white balance correction, illumination normalization, noise suppression, and resolution normalization.

6. The control method of the stool state auxiliary assessment system as described in claim 2, characterized in that, The image acquisition module includes one or more of the following: a built-in camera unit in a mobile terminal, a camera unit in a wearable device, an external camera module, and a camera unit in a medical testing device.

7. The control method of the stool state auxiliary assessment system as described in claim 2, characterized in that, The structure of the stool image feature extraction model includes one or more of the following: Convolutional Neural Network; Visual Transformer; Convolutional-Transformer hybrid neural networks; Multi-branch deep neural networks based on attention mechanisms; Multi-scale feature pyramid network.

8. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The step of constructing input features based on the defecation features and the text vector includes: The text vector and the defecation features are encoded into a preset format to obtain standardized information; Prompt information is constructed based on the standardized information and the preset prompt word template; The prompt information is used as the input feature.

9. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The step of constructing input features based on the defecation features and the text vector includes: The defecation features are encoded into a vector format to obtain a defecation vector; The input features are obtained by concatenating the defecation vector with the text vector.

10. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The step of constructing input features based on the defecation features and the text vector further includes: The defecation features are encoded into a vector format to obtain a defecation vector; The defecation vector and the text vector are input into a multimodal Transformer, and the cross-attention weights between the defecation vector and the text vector are calculated through the cross-modal attention layer of the multimodal Transformer. Based on the cross-attention weights, the defecation vector and the text vector are weighted respectively; The weighted defecation vector is concatenated with the text vector to obtain the input features.

11. The control method of the stool state auxiliary assessment system as described in claim 10, characterized in that, The step of calculating the cross-attention weights between the defecation vector and the text vector through the cross-modal attention layer of the multimodal Transformer further includes: Using the defecation vector as the query vector and the text vector as the key and value vectors, calculate the first cross-attention weight of the defecation vector on the text vector; Using the text vector as the query vector and the defecation vector as the key and value vectors, calculate the second cross-attention weight of the text vector on the defecation vector.

12. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The step of constructing input features based on the defecation features and the text vector includes: Determine the mapping nodes of the text vector and the defecation features in a preset medical knowledge graph; In the preset medical knowledge graph, a preset number of associated nodes and associated edges connected to the mapping node are determined to obtain a sub-graph. The sub-graph is aggregated using a pre-trained graph neural network model, and the resulting node feature vectors are used as the input features.

13. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The multidimensional medical record includes one or more of the following: basic information, feeding information, accompanying symptoms, past medical history / medication history, examination information, physician's conclusions, and follow-up conclusions.

14. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The auxiliary assessment results include one or more of the following: risk warning level, digestive state type, candidate causes of suspicion corresponding to the digestive state type, and evidence points corresponding to the candidate causes of suspicion.

15. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, The large language model includes a general large language model and a medical vertical large language model.

16. The control method of the stool state auxiliary assessment system as described in claim 1, characterized in that, Before the step of inputting the input features into the large language model to determine the auxiliary evaluation result associated with the target evaluation object, the method further includes: A set of instructions is constructed based on medical record information and bowel information from medical record samples; A response set is constructed based on the assessment labels in the medical record sample that correspond to the medical record information and the bowel information; The instruction set and the response set are input as instruction-response pairs into the base model, and the base model is trained to update the linear layer parameters of the base model, thereby obtaining the large language model.

17. The control method of the stool state auxiliary assessment system as described in claim 16, characterized in that, Before the step of inputting the instruction set and the response set as instruction-response pairs into the base model and training the base model, the method further includes: The sum of the original weight matrix of each target linear layer in the base model and the preset low-rank increment matrix is ​​used as the weight matrix of the target linear layer.

18. The control method of the stool state auxiliary assessment system as described in claim 17, characterized in that, The original weight matrix includes the query projection matrix, key projection matrix, and value projection matrix in the attention module, and / or the up and down projection matrices in the feedforward neural network, wherein the rank of the low-rank increment matrix is ​​less than the dimension of the hidden layer feature vector of the base model.

19. The control method of the stool state auxiliary assessment system as described in claim 18, characterized in that, The steps of inputting the instruction set and the response set as instruction-response pairs into the base model, training the base model to update the linear layer parameters of the base model, and obtaining the large language model include: The instruction-response pair is input into the base model to obtain the predicted evaluation label output by the base model, which is determined based on the weight matrix and the feature vector in the instruction set. Determine the lexical cross-entropy loss between the predicted evaluation label and the feature vector in the response set; The low-rank increment matrix is ​​updated based on the lexical cross-entropy loss to update the linear layer parameters of the base model, thereby obtaining the large language model.

20. The control method of the stool state auxiliary assessment system as described in claim 19, characterized in that, The step of updating the low-rank increment matrix based on the lexical cross-entropy loss to update the linear layer parameters of the base model and obtain the large language model includes: The gradient value of the lexical cross-entropy loss relative to the low-rank increment matrix is ​​determined using a preset backpropagation algorithm. Based on the low-rank increment matrix, the gradient value, and the hyperparameters of the base model, a new low-rank increment matrix is ​​determined; The sum of the new low-rank increment matrix and the weight matrix is ​​used as the new weight matrix of the target linear layer to update the linear layer parameters of the base model. The base model after updating the linear layer parameters is used as the large language model.

21. The control method of the stool state auxiliary assessment system as described in any one of claims 16 to 20, characterized in that, The step of inputting the input features into a large language model to determine the auxiliary evaluation result associated with the target evaluation object includes: The input features are fed into the target linear layer in the large language model, the target linear layer including an attention module and a feedforward network; In each target linear layer, the feature vector corresponding to the linear layer is determined based on the output vector of the preceding linear layer and the weight matrix of the attention module, and the output vector of the linear layer is determined based on the feature vector and the upper and lower projection matrices of the feedforward network. The output vector of the last target linear layer is mapped to the probability value of the evaluation label through the output layer of the large language model; Based on the probability value, the auxiliary evaluation result corresponding to the input feature is determined.

22. A stool condition auxiliary assessment system, characterized in that, The stool condition auxiliary assessment system includes: The visual analysis module is used to extract defecation features based on stool images; The medical record parsing module is used to extract text vectors based on multidimensional medical record records; A multimodal fusion module is used to construct input features based on the defecation features and the text vector; A large language model is used to determine the auxiliary evaluation results associated with the target evaluation object based on the input features.

23. A control device for a stool condition auxiliary assessment system, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the fecal state auxiliary assessment system as described in any one of claims 1 to 21.

24. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control method of the defecation state auxiliary assessment system as described in any one of claims 1 to 21.

Images