Uric acid data acquisition and anomaly detection method and device based on multi-modal fusion

Abstract

The embodiment of the invention discloses a uric acid data acquisition and anomaly detection method and device based on multi-modal fusion. A specific embodiment of the method comprises the following steps: controlling a medical data query end to query a historical uric acid detection data set corresponding to uric acid detail retrieval information; performing image enhancement processing on the initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; performing uric acid area identification on the enhanced uric acid detection image to obtain a uric acid identification area image group; optimizing each uric acid recognition area image in the uric acid recognition area image group to generate an optimized uric acid recognition area image, and obtaining an optimized uric acid recognition area image group; and inputting the historical uric acid detection data set and the optimized uric acid recognition area image group into a pre-trained multi-mode uric acid anomaly detection model to generate a uric acid anomaly detection result. According to the embodiment, the detection accuracy is improved, and accurate dynamic detection of uric acid abnormity is realized.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of uric acid data collection and anomaly detection, and specifically to a method and device for uric acid data collection and anomaly detection based on multimodal fusion. Background Art

[0002] In the field of medical testing, early and accurate diagnosis of uric acid abnormalities (such as hyperuricemia and gout) is crucial for preventing joint damage and renal failure. Existing technologies have three major bottlenecks:

[0003] First, data fragmentation: uric acid images (CT / ultrasound), biochemical indicators (blood uric acid levels), and historical medical records are scattered across different systems, lacking the ability to perform dynamic correlation analysis.

[0004] Second, imaging misdetection: Traditional imaging equipment has a low recognition rate (about 37%) for tiny urate crystals (<3mm) and is easily interfered by artifacts;

[0005] Third, response delay: Detection results rely on manual review, which cannot provide real-time warning of abnormal trends and delays intervention opportunities.

[0006] Although there is a uric acid recognition model based on deep learning, the problems of multi-source data fusion and asynchronous monitoring have not been solved, resulting in a high false positive rate and insufficient system robustness.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0008] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0009] Some embodiments of the present disclosure propose a uric acid data collection and abnormality detection method and device based on multimodal fusion to solve the technical problems mentioned in the above background technology section.

[0010] In the first aspect, some embodiments of the present disclosure provide a uric acid data collection and abnormality detection method based on multimodal fusion, the method comprising: controlling a uric acid detection device to collect an initial uric acid detection image of a target user; generating corresponding uric acid detail retrieval information based on historical uric acid retrieval information corresponding to the above-mentioned target user, and generating uric acid retrieval association information based on retrieval bytecode information; controlling the above-mentioned medical data query monitoring terminal to monitor the retrieval performance of the above-mentioned historical uric acid retrieval information based on asynchronous retrieval operation information, wherein the above-mentioned asynchronous retrieval operation information corresponds to the above-mentioned uric acid retrieval association information; controlling the medical data query terminal to query the above-mentioned uric acid retrieval information corresponding to the above-mentioned uric acid retrieval information. A historical uric acid detection dataset with acid detail retrieval information; performing image enhancement processing on the above-mentioned initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; performing uric acid region recognition on the above-mentioned enhanced uric acid detection image to obtain a uric acid recognition region image group; optimizing each uric acid recognition region image in the above-mentioned uric acid recognition region image group to generate an optimized uric acid recognition region image to obtain an optimized uric acid recognition region image group; inputting the above-mentioned historical uric acid detection dataset and the above-mentioned optimized uric acid recognition region image group into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

[0011] In the second aspect, some embodiments of the present disclosure provide a uric acid data collection and abnormality detection device based on multimodal fusion, the device comprising: an acquisition unit, configured to control the uric acid detection device to collect the initial uric acid detection image of the target user; a generation unit, configured to generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the above-mentioned target user, and generate uric acid retrieval related information based on the retrieval bytecode information; a control unit, configured to control the above-mentioned medical data query monitoring end to monitor the retrieval performance of the above-mentioned historical uric acid retrieval information based on asynchronous retrieval operation information, wherein the above-mentioned asynchronous retrieval operation information corresponds to the above-mentioned uric acid retrieval related information; a query unit, configured to control the medical data query end to query the uric acid retrieval information corresponding to the above-mentioned uric acid The invention relates to a historical uric acid detection dataset for uric acid detail retrieval information; an enhancement unit is configured to perform image enhancement processing on the above-mentioned initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; an identification unit is configured to perform uric acid region identification on the above-mentioned enhanced uric acid detection image to obtain a uric acid identification region image group; an optimization unit is configured to optimize each uric acid identification region image in the above-mentioned uric acid identification region image group to generate an optimized uric acid identification region image to obtain an optimized uric acid identification region image group; an input unit is configured to input the above-mentioned historical uric acid detection dataset and the above-mentioned optimized uric acid identification region image group into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0014] The above-mentioned various embodiments of the present application have the following beneficial effects: through the uric acid data collection and abnormality detection method based on multimodal fusion of some embodiments of the present disclosure, multimodal fusion and intelligent asynchronous monitoring are realized, and accurate dynamic detection of uric acid abnormalities is realized. Specifically, first, the uric acid detection device is controlled to collect the initial uric acid detection image of the target user; based on the historical uric acid retrieval information corresponding to the above-mentioned target user, the corresponding uric acid detail retrieval information is generated, and based on the retrieval bytecode information, uric acid retrieval related information is generated; based on the asynchronous retrieval operation information, the above-mentioned medical data query monitoring terminal is controlled to monitor the retrieval performance of the above-mentioned historical uric acid retrieval information, wherein the above-mentioned asynchronous retrieval operation information corresponds to the above-mentioned uric acid retrieval related information; the medical data query terminal is controlled to query the historical uric acid detection data set corresponding to the above-mentioned uric acid detail retrieval information. Thus, the query performance is dynamically monitored based on the retrieval bytecode information, and the data retrieval delay is shortened; the asynchronous operation mechanism processes image enhancement and data query in parallel, which greatly speeds up the overall detection process. Then, according to the pre-trained uric acid detection image enhancement model, the above-mentioned initial uric acid detection image is subjected to image enhancement processing to generate an enhanced uric acid detection image; uric acid region recognition is performed on the above-mentioned enhanced uric acid detection image to obtain a uric acid recognition region image group; each uric acid recognition region image in the above-mentioned uric acid recognition region image group is optimized to generate an optimized uric acid recognition region image, thereby obtaining an optimized uric acid recognition region image group. Thus, the historical uric acid data and real-time image features are integrated to construct a cross-modal association model, thereby improving the accuracy of microcrystal recognition; the image enhancement model optimizes low-contrast areas and reduces artifact interference. Finally, the above-mentioned historical uric acid detection dataset and the above-mentioned optimized uric acid recognition region image group are input into the pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result. Thus, the multimodal abnormality detection model integrates the uric acid region image group and the historical dataset, greatly improving the detection accuracy and realizing accurate dynamic detection of uric acid abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flowchart of some embodiments of the uric acid data collection and abnormality detection method based on multimodal fusion according to the present disclosure;

[0017] Figure 2 Schematic diagram of the model structure of the uric acid detection image enhancement model in the uric acid data acquisition and abnormality detection method based on multimodal fusion disclosed in the present invention;

[0018] Figure 3 Schematic diagram of the structure of some embodiments of the uric acid data acquisition and abnormality detection device based on multimodal fusion according to the present disclosure;

[0019] Figure 4 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0021] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0023] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0025] Before performing any operations involving the collection, storage, and use of user personal information (such as uric acid test images) in this disclosure, the relevant organizations or individuals must fulfill their obligations, including conducting personal information security impact assessments, fulfilling their obligations to inform the personal information subjects, and obtaining the prior authorization and consent of the personal information subjects.

[0026] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0027] Figure 1 The flowchart 100 of some embodiments of the uric acid data collection and abnormality detection method based on multimodal fusion according to the present disclosure is shown. The uric acid data collection and abnormality detection method based on multimodal fusion includes the following steps:

[0028] Step 101: Control the uric acid testing device to collect an initial uric acid testing image of the target user.

[0029] In some embodiments, the execution entity (e.g., a computing device) of the multimodal fusion-based uric acid data collection and abnormality detection method can control a uric acid testing device to collect an initial uric acid test image of a target user. The uric acid testing device can refer to a device that is communicatively connected to the execution entity and is used to collect uric acid data. For example, the uric acid testing device can be a medical ultrasound diagnostic instrument, which mainly consists of two parts: the device host and the ultrasound probe. The ultrasound diagnostic instrument host primarily processes and displays the signals received from the probe. The ultrasound probe can transmit and receive ultrasound, perform electroacoustic signal conversion, convert electrical signals sent by the host into high-frequency oscillating ultrasound signals, and convert ultrasound signals reflected from tissues and organs into electrical signals for display on the host display. For another example, the uric acid testing device can be a urine analyzer, an automated instrument for detecting uric acid in urine using photoelectric colorimetric analysis technology to determine certain chemical components in urine. The initial uric acid test image can refer to an ultrasound image of the target user's uric acid level. The target user can refer to the patient whose uric acid level is to be tested. For example, the execution entity may control a medical ultrasonic diagnostic apparatus to perform ultrasonic detection on the target user to obtain an initial uric acid detection image.

[0030] Step 102: Generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the target user, and generate uric acid retrieval related information based on the retrieval bytecode information.

[0031] In some embodiments, the execution entity may generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the target user, and generate uric acid retrieval-related information based on the retrieval bytecode information. The historical uric acid retrieval information may represent the operation information for retrieving the target user's historical uric acid data. For example, the historical uric acid retrieval information may be the operation of a medical staff entering a search statement on the search page. For example, the search statement may be "query the target user's historical uric acid data."

[0032] In some optional implementations of some embodiments, the execution entity may generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the target user through the following steps:

[0033] The first step is to determine, based on the historical uric acid search information, the search symptom logic information corresponding to the historical uric acid search information. The search symptom logic information may represent the search symptom logic corresponding to the historical uric acid search information. For example, the execution entity may determine the search type corresponding to the historical uric acid search information. Then, based on the search type, the search symptom logic information corresponding to the historical uric acid search information is determined from a preset uric acid symptom logic type table. The uric acid symptom logic type table may represent the correspondence between the search type and the symptom logic information.

[0034] The second step is to generate combined search formula information corresponding to the historical uric acid search information based on the aforementioned search symptom logic information. The combined search formula information may represent a combined search formula. For example, the execution entity may use a dynamic DSL method to generate combined search formula information corresponding to the historical uric acid search information based on the aforementioned search symptom logic information.

[0035] The third step is to generate a call search formula information corresponding to the historical uric acid search information based on the combined search formula information. For example, the execution entity may determine the corresponding API interface call method as the call search formula information corresponding to the historical uric acid search information.

[0036] The fourth step is to combine the above-mentioned symptom search logic information, the above-mentioned combined search formula sentence information, and the above-mentioned call search formula information into uric acid detail search information corresponding to the above-mentioned historical uric acid search information.

[0037] The uric acid search-related information may represent search data associated with the historical uric acid search information. The search bytecode information may represent the bytecode of the source code corresponding to the search formula information. The uric acid search-related information may include: search conditions, user ID, and search time. For example, the execution entity may capture the uric acid search-related information corresponding to the query bytecode information using a Spring AOP method.

[0038] Step 103: Based on the asynchronous retrieval operation information, the medical data query monitoring terminal is controlled to monitor the retrieval performance of the historical uric acid retrieval information.

[0039] In some embodiments, the execution entity may control the medical data query monitoring terminal to monitor the retrieval performance of the historical uric acid retrieval information based on the asynchronous retrieval operation information. The asynchronous retrieval operation information corresponds to the uric acid retrieval association information. The asynchronous retrieval operation information may represent the set asynchronous retrieval operation information. For example, the asynchronous retrieval operation information includes server information, and the IP address represented by the server information is a preset IP address. The asynchronous retrieval operation information may represent an asynchronous operation triggered by bytecode enhancement technology. The asynchronous retrieval operation information may include: DSL query statement, user ID, time, and server IP. For example, the execution entity may determine the preset asynchronous retrieval operation setting information as the asynchronous retrieval operation information corresponding to the uric acid retrieval association information. For example, the execution entity may monitor the retrieval performance of the historical uric acid retrieval information based on the asynchronous retrieval operation information using a retrieval performance tool. For example, the retrieval performance tool may be an APM (Application Performance Management) tool. The medical data query monitoring terminal may be a terminal that monitors the retrieval query capabilities of a database storing medical data.

[0040] In some optional implementations of some embodiments, the execution entity may monitor the retrieval performance of the historical uric acid retrieval information through the following steps:

[0041] The first step is to control the medical data query monitoring terminal to monitor the asynchronous retrieval operation information to determine whether the asynchronous retrieval operation information meets a preset monitoring condition. For example, the execution entity may control the medical data query monitoring terminal to monitor and process the asynchronous retrieval operation information through a listener. The monitoring condition may be that the server ID included in the asynchronous retrieval operation information is the same as a preset server ID.

[0042] In the second step, in response to determining that the asynchronous retrieval operation information satisfies the monitoring conditions, field extraction is performed on the asynchronous retrieval operation information to obtain the respective retrieval field information corresponding to the asynchronous retrieval operation information. The respective retrieval field information may represent the respective retrieval fields extracted from the asynchronous retrieval operation information. For example, the execution entity may control the medical data query monitoring terminal to perform field extraction processing on the asynchronous retrieval operation information using a field parsing and extraction tool to obtain the respective retrieval field information corresponding to the asynchronous retrieval operation information. For example, the field extraction and parsing tool may be a JSONPath tool.

[0043] The third step is to combine the above search field information to obtain uric acid search embedding information. For example, the execution entity can use the general monitoring embedding SDK to assemble the above search field information to obtain uric acid search embedding information.

[0044] The fourth step is to add the above uric acid search embedding information to a preset asynchronous search information queue. The above asynchronous search information queue can be a Message Queue.

[0045] The fifth step, in response to determining that the above-mentioned uric acid retrieval embedding information meets the preset embedding storage condition, generates corresponding retrieval performance monitoring information based on the above-mentioned uric acid retrieval embedding information to monitor and process the retrieval performance of the above-mentioned historical uric acid retrieval information. Among them, the above-mentioned preset embedding storage condition can be that the order of the above-mentioned uric acid retrieval embedding information in the above-mentioned preset retrieval information queue is a preset order or the priority of the above-mentioned uric acid retrieval embedding information is a preset priority. The above-mentioned preset order can be the first one. The above-mentioned preset priority can be the first level. The above-mentioned retrieval performance monitoring information can characterize the monitored retrieval behavior or the monitored retrieval performance.

[0046] In this way, the flexibility and maintainability of retrieval performance monitoring can be improved, and the difficulty of migrating retrieval performance monitoring can be reduced, as well as the difficulty of retrieval performance monitoring and the consumed computing resources can be reduced.

[0047] Step 104 : Control the medical data query terminal to query the historical uric acid test data set corresponding to the above uric acid detail retrieval information.

[0048] In some embodiments, the above-mentioned execution entity can control the medical data query end to query the historical uric acid detection data set corresponding to the above-mentioned uric acid detail retrieval information. The above-mentioned historical uric acid detection data set can represent the retrieval results corresponding to the above-mentioned uric acid detail retrieval information. In practice, the above-mentioned execution entity can control the above-mentioned medical data query end to input the above-mentioned uric acid detail retrieval information into the query search engine to obtain the historical uric acid detection data set. For example, the query search engine can be Elasticsearch. Historical uric acid detection data can refer to the uric acid retrieval data of the target user at a historical time point, which can include image data, test results, etc.

[0049] Step 105 : performing image enhancement processing on the initial uric acid detection image according to the pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image.

[0050] In some embodiments, the execution entity may perform image enhancement processing on the initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image. For example, the uric acid detection image enhancement model may be an LLNet (Low Light Network) model. The uric acid detection image enhancement model includes: a surface image feature extraction network, a multi-level image feature encoding network, and an image enhancement network. The surface image feature extraction network includes: a surface cross encoding layer, the multi-level image feature encoding network includes: a first cross encoder, a second cross encoder, and a third cross encoder; the first cross encoder, the second cross encoder, and the third cross encoder are connected in series; the image enhancement network includes: a first enhancement decoder, a second enhancement decoder, and a third enhancement decoder; the first enhancement decoder, the second enhancement decoder, and the third enhancement decoder are connected in series; a first attention fusion network layer is provided between the first cross encoder and the first enhancement decoder; a second attention fusion network layer is provided between the second cross encoder and the second enhancement decoder; a third attention fusion network layer is provided between the third cross encoder and the third enhancement decoder; and an encoding network is provided after the third attention fusion network layer.

[0051] In some optional implementations of some embodiments, the execution entity may perform image enhancement processing on the initial uric acid detection image through the following steps:

[0052] S1. Performing surface feature extraction on the initial uric acid detection image through the surface image feature extraction network included in the uric acid detection image enhancement model to generate surface uric acid detection image features.

[0053] S2, feature encoding the surface uric acid detection image features through the multi-level image feature encoding network included in the uric acid detection image enhancement model to generate uric acid detection encoded image features.

[0054] S3, performing image enhancement on the uric acid detection encoding image features through the image enhancement network included in the uric acid detection image enhancement model to generate an enhanced uric acid detection image.

[0055] For example, see Figure 2 Schematic diagram of the structure of the uric acid detection image enhancement model, in which the input of the surface cross-coding layer is the initial uric acid detection image. The input of the first cross-encoder is the output of the surface cross-coding layer. The output of the first cross-encoder is sent to the second cross-encoder, the first attention fusion network layer, and the first enhancement decoder. The output of the second cross-encoder is sent to the third cross-encoder, the second attention fusion network layer, and the second enhancement decoder; the output of the third cross-encoder is sent to the third attention fusion network layer and the third enhancement decoder. The output of the first enhancement decoder is sent to the first attention fusion network layer and the second enhancement decoder; the output of the second enhancement decoder is sent to the second attention fusion network layer and the third enhancement decoder; the output of the third enhancement decoder is sent to the third attention fusion network layer.

[0056] The surface cross-encoding layer and the encoding network each include three encoding layers. The encoding layer includes: a normalization layer, a convolution layer (with a 1×1 kernel), two multi-head attention mechanism modules, a normalization layer, and a feedforward network. The first cross-encoder, the second cross-encoder, and the third cross-encoder include two sub-encoding modules and a downsampling network. The two multi-head attention mechanism modules include: a multi-scale convolution layer (Multi-scale Convolution) and a multi-head attention mechanism layer (Multi-head Attention). The first enhanced decoder, the second enhanced decoder, and the third enhanced decoder include: an upsampling network and three sub-encoding modules.

[0057] The first, second, and third attention fusion layers each include three average pooling layers, three multilayer perceptrons, a Sigmoid activation function, and a ReLU activation function. The three average pooling layers and three multilayer perceptrons are arranged symmetrically.

[0058] Thus, the surface image feature extraction network extracts surface features from the image. The features are then encoded and decoded using a symmetrically designed encoder and decoder. The surface cross-coding layer extracts surface features, and the encoding network suppresses noise in the output features. Furthermore, to reduce the amount of feature processing, the cross-encoder decomposes feature attention into horizontal and vertical dimensions, using a cross-design to correlate the results, thereby improving image enhancement capabilities.

[0059] Step 106: Perform uric acid region recognition on the enhanced uric acid detection image to obtain a uric acid recognition region image group.

[0060] In some embodiments, the execution entity may perform uric acid region recognition on the enhanced uric acid detection image to obtain a uric acid recognition region image set. For example, each uric acid region in the enhanced uric acid detection image may be identified using dual-energy CT or plain X-ray (KUB).

[0061] For another example, the uric acid region can be identified on the enhanced uric acid detection image through dual-energy CT (DECT) recognition model, three-dimensional reconstruction and quantitative analysis model, traditional CT density analysis model, ultrasound image feature model, etc., to obtain a uric acid recognition region image group.

[0062] The dual-energy CT (DECT) identification model utilizes 80kVp and 140kVp dual-energy scans to detect differences in attenuation values ​​for uric acid stones at low and high energy spectra. Uric acid exhibits a unique attenuation slope in the energy spectrum due to its low atomic number (effective atomic number 6.8–7.8).

[0063] The dual-energy CT (DECT) recognition model utilizes the following methods: 1. Material separation map: Post-processing software generates a color-coded map specifically for uric acid (e.g., green for uric acid, and different colors for other components). 2. Dual-energy ratio analysis: Calculates the attenuation ratio between high and low energy scans (the ratio for uric acid is significantly lower than for calcium stones).

[0064] Technical process of 3D reconstruction and quantitative analysis model: Based on dual-source CT data, use Mimics and other software to generate a 3D model of tophi, annotating the location, volume, and proximity to blood vessels / nerves.

[0065] The principle of the traditional CT density analysis model: the density of uric acid stones is significantly lower than that of calcium stones, with a typical CT value of 300–500HU (calcium stones >1000HU); the identification method is threshold segmentation (such as setting HU <500 areas as suspected uric acid stones).

[0066] Characteristic signs of the ultrasound imaging feature model: 1. Double contour sign: urate deposits on the surface of articular cartilage, forming high-echo lines; 2. Aggregates: punctate / clustered high-echo areas with acoustic shadows in the joint cavity.

[0067] In some optional implementations of some embodiments, the execution entity may perform uric acid region recognition on the enhanced uric acid detection image through the following steps:

[0068] The first step is to mark each uric acid stone area in the enhanced uric acid test image to obtain a marked uric acid test image. The uric acid stone area can be marked in the enhanced uric acid test image by a radiologist using manual delineation, automatic, semi-automatic, or manual methods to obtain a marked uric acid test image.

[0069] In the second step, two-dimensional image segmentation is performed on the annotated uric acid test images to obtain a two-dimensional uric acid test segmentation image set. For example, the annotated uric acid test images can be input into a trained image segmentation model to obtain a two-dimensional uric acid test segmentation image set. The above image segmentation model is used to segment two-dimensional images. The above image segmentation model may refer to a Swin Transformer U-type network (Swin Transformer Unet, Swin-Unet) model for medical image segmentation. The above image segmentation model is constructed by a Swin transformer (Shifted Window Transformer) model and a Unet structure. The Swin-Unet model deeply integrates the global context modeling capability of the Swin Transformer and the detail retention advantage of the U-Net. The Unet model is a convolutional neural network for image segmentation that adopts a symmetrical encoder-decoder structure.

[0070] In the third step, for each two-dimensional uric acid detection segmentation image in the above two-dimensional uric acid detection segmentation image set, perform the following processing steps:

[0071] S1, extracting high semantic features from the two-dimensional uric acid detection segmented image to generate high semantic features. The execution subject may extract high semantic features from the two-dimensional uric acid detection segmented image through an encoder to generate high semantic features.

[0072] S2: Perform upsampling feature extraction on the two-dimensional uric acid detection segmentation image to generate upsampling features. The execution subject may perform upsampling feature extraction on the two-dimensional uric acid detection segmentation image through a transposed convolution layer to generate upsampling features.

[0073] S3, concatenating the high semantic features and the upsampled features to generate a concatenated feature. As an example, the execution entity may concatenate the high semantic features and the upsampled features using Python on the same channel dimension to generate a concatenated feature.

[0074] S4, performing spatial weight image generation on the splicing features to generate a spatial weight image. As an example, the execution entity may perform spatial weight image generation on the splicing features through a convolutional layer to generate a spatial weight image.

[0075] S5, performing weighted processing on the spatial weight image to generate a weighted feature image. As an example, the execution entity may multiply the spatial weight image by the splicing feature element by element to generate the weighted feature image.

[0076] S6: Decode the weighted feature image to generate a decoded uric acid feature map. As an example, the execution subject may decode the weighted feature image through a decoder to generate a decoded uric acid feature map.

[0077] The fourth step is to determine the obtained decoded uric acid feature maps as a uric acid region image group.

[0078] Therefore, the readability of the uric acid region image can be improved through image conversion and two-dimensional segmentation, thereby reducing the probability of losing key information and improving the robustness of the uric acid region image.

[0079] Step 107 : Optimize each uric acid recognition region image in the uric acid recognition region image group to generate an optimized uric acid recognition region image, thereby obtaining an optimized uric acid recognition region image group.

[0080] In some embodiments, the execution entity may optimize each uric acid recognition region image in the uric acid recognition region image group to generate an optimized uric acid recognition region image, thereby obtaining an optimized uric acid recognition region image group. For example, the execution entity may optimize the uric acid recognition region image through the following steps, which may include: contrast adjustment, brightness / color correction; edge sharpening (Laplacian operator, unsharp mask); and various optimization methods such as medical imaging-specific optimization (CT / MRI enhancement, ultrasound image optimization).

[0081] In some optional implementations of some embodiments, the execution entity may optimize each uric acid recognition region image in the uric acid recognition region image group through the following steps to generate an optimized uric acid recognition region image:

[0082] The first step is to optimize the subregion boundaries of the uric acid recognition region image through morphological operations to obtain a first optimized uric acid recognition region image. Morphological operations may include, but are not limited to, dilation, erosion, opening, and closing operations. For example, opening operations can be used to remove small noise and isolated areas and smooth subregion boundaries, while closing operations can be used to fill small holes within subregions to ensure subregion continuity.

[0083] The second step is to perform topological repair on the first optimized uric acid recognition region image to obtain a repaired uric acid recognition region image as the optimized uric acid recognition region image. For example, a closed operation priority and an erosion-dilation combination can be used to smooth the boundaries of the first optimized uric acid recognition region image.

[0084] Step 108: Input the above historical uric acid detection dataset and the above optimized uric acid recognition area image group into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

[0085] In some embodiments, the execution entity may input the historical uric acid test data set and the optimized uric acid recognition region image set into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result. For example, the multimodal uric acid abnormality detection model may be a multimodal neural network model that comprehensively detects whether uric acid is abnormal based on the user's historical uric acid test data and the current uric acid recognition region image. For example, the multimodal uric acid abnormality detection model may be a multimodal machine learning prediction model.

[0086] The technical principles of the multimodal machine learning prediction model are as follows: integrating genetic data (such as SNPs) with clinical indicators (BMI, blood lipids, renal function, etc.), generating a comprehensive risk score (such as ISHUA) through a stacked machine learning architecture (base model + meta-model), enabling early screening of hyperuricemia (HUA) and prediction of complications; the base model: seven classifiers, including LightGBM and XGBoost, process genetic and clinical features in parallel; the meta-model: XGBoost integrates the output of the base model to generate the final prediction probability; feature screening: LASSO regression selects 1,378 key genetic markers from 38,277 SNPs and combines them with 10 clinical indicators.

[0087] Optionally, the multimodal uric acid abnormality detection model can be trained by the following steps:

[0088] The first step is to obtain a historical uric acid sign detection data set and a corresponding detection label set. The historical uric acid sign detection data includes: uric acid sign detection data and a historical uric acid identification area image group, and one historical uric acid sign detection data corresponds to one detection label. The detection label may include a data label and an image label. The data label corresponds to the uric acid sign detection data. The image label corresponds to the historical uric acid identification area image group. The data label may indicate whether the user corresponding to the uric acid sign detection data will have abnormal uric acid at a future time node. The image label indicates whether the corresponding historical uric acid identification area image group has abnormal uric acid.

[0089] The second step is to preprocess each uric acid sign detection data in the above historical uric acid sign detection dataset to generate preprocessed uric acid sign detection data, thereby obtaining a preprocessed uric acid sign detection dataset. Data preprocessing may include: data normalization, data completion, and data enhancement.

[0090] The third step is to determine the network structure of the initial multimodal uric acid abnormality detection model. The above-mentioned initial multimodal uric acid abnormality detection model includes: an initial uric acid sign detection data detection model and an initial uric acid identification area image detection model. The initial uric acid sign detection data detection model may be an untrained uric acid sign detection data detection model. The uric acid sign detection data detection model is used to identify and predict uric acid sign detection data, and may include: an input layer, a multi-level convolution layer (Conv2D+ReLU), a pooling layer (MaxPooling), and a classification output layer. The initial uric acid identification area image detection model may be an untrained uric acid identification area image detection model. For example, the uric acid identification area image detection model may be a fusion model of a dual-energy CT urate crystal recognition model and an ultrasound image feature recognition model. The ultrasound image feature recognition model may include: an input layer, a convolution layer (primary convolution (Conv2D+ReLU), a pooling layer (MaxPooling)), and a fully connected layer.

[0091] The fourth step is to perform model training on the initial detection data semantic extraction model based on the above-mentioned preprocessed uric acid sign detection data set to generate a detection data semantic extraction model. The above-mentioned detection data semantic extraction model includes: a feature extraction layer for extracting cross-feature information between each sign data in the uric acid sign detection data. The initial detection data semantic extraction model may be a semantic extraction model that has not yet been trained. The semantic extraction model may be a neural network model that learns the detection content corresponding to the uric acid sign detection task. For example, the semantic extraction model may be a multi-layer serial encoding layer (for example, a plurality of serial downsampling layers). The above-mentioned semantic extraction model includes: a feature extraction layer for extracting cross-feature information between each sign data.

[0092] The fifth step is to perform model training on the initial uric acid sign detection data detection model based on the above-mentioned preprocessed uric acid sign detection data set and the above-mentioned detection label set to generate a uric acid sign detection data detection model. The above-mentioned uric acid sign detection data detection model includes: the feature extraction layer included in the above-mentioned detection data semantic extraction model. Here, the model training method can refer to the training method of the deep neural network model, which will not be repeated here. For example, during the model training process, the hinge loss function or the cross entropy loss function can be used to identify the loss of the model. The optimization method can be optimized using the stochastic gradient descent method.

[0093] In the sixth step, the initial uric acid recognition area image detection model is trained according to each historical uric acid recognition area image group included in the above-mentioned historical uric acid sign detection data set to obtain a trained uric acid recognition area image detection model. Here, the method of model training can refer to the training method of the deep neural network model, which will not be repeated here. For example, during the model training process, a hinge loss loss function or a cross entropy loss function can be used to identify the model loss. The optimization method can be optimized using the stochastic gradient descent method.

[0094] The seventh step is to integrate the above-mentioned uric acid sign detection data detection model with the above-mentioned uric acid recognition area image detection model into a trained multimodal uric acid abnormality detection model.

[0095] Therefore, by using the pre-trained feature extraction layer as the feature extraction module in the initial uric acid sign detection data detection model, the cross-content features of the detection content under each sign data can be learned, which is convenient for efficient model training of the multimodal uric acid abnormality detection model, improves the understanding of the sign data content and shortens the training time. In summary, by pre-training the initial detection data semantic extraction model, a feature extraction layer that extracts cross-feature information between each parameter in the sign data can be obtained. By using the feature extraction layer as the feature extraction module of the initial uric acid sign detection data detection model, the initial uric acid sign detection data detection model can extract the cross-correlation information of the parameter content between each sign data, thereby ensuring the model training accuracy of the multimodal uric acid abnormality detection model and shortening the training time.

[0096] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a uric acid data acquisition and abnormality detection device based on multimodal fusion. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the uric acid data collection and abnormality detection device based on multimodal fusion can be specifically applied to various electronic devices.

[0097] like Figure 3As shown, some embodiments of the uric acid data collection and anomaly detection device 300 based on multimodal fusion include: an acquisition unit 301, a generation unit 302, a control unit 303, a query unit 304, an enhancement unit 305, a recognition unit 306, an optimization unit 307 and an input unit 308. The acquisition unit 301 is configured to control the uric acid detection device to collect the initial uric acid detection image of the target user; the generation unit 302 is configured to generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the above-mentioned target user, and generate uric acid retrieval association information based on the retrieval bytecode information; the control unit 303 is configured to control the above-mentioned medical data query monitoring end to monitor the retrieval performance of the above-mentioned historical uric acid retrieval information based on the asynchronous retrieval operation information, wherein the above-mentioned asynchronous retrieval operation information corresponds to the above-mentioned uric acid retrieval association information; the query unit 304 is configured to control the medical data query end to query the historical uric acid detection data set corresponding to the above-mentioned uric acid detail retrieval information; the enhancement unit 305 is configured to perform image enhancement processing on the above-mentioned initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; the recognition unit 306 is configured to perform uric acid region recognition on the above-mentioned enhanced uric acid detection image to obtain a uric acid recognition region image group; the optimization unit 307 is configured to optimize each uric acid recognition region image in the above-mentioned uric acid recognition region image group to generate an optimized uric acid recognition region image to obtain an optimized uric acid recognition region image group; the input unit 308 is configured to input the above-mentioned historical uric acid detection data set and the above-mentioned optimized uric acid recognition region image group into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

[0098] It is understandable that the various units recorded in the uric acid data acquisition and abnormality detection device 300 based on multimodal fusion and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the uric acid data acquisition and abnormality detection device 300 based on multimodal fusion and the units contained therein, and will not be repeated here.

[0099] Reference below Figure 4 , which shows a schematic structural diagram of an electronic device (such as a computing device) suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 4As shown, the computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can enable the processor to execute any one of the above-mentioned uric acid data collection and anomaly detection methods based on multimodal fusion. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, can enable the processor to execute any one of the above-mentioned uric acid data collection and anomaly detection methods based on multimodal fusion. The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0100] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0101] In one embodiment, the processor is configured to execute a computer program stored in a memory to implement the following steps: controlling a uric acid detection device to collect an initial uric acid detection image of a target user; generating corresponding uric acid detail retrieval information based on historical uric acid retrieval information corresponding to the target user, and generating uric acid retrieval association information based on retrieval bytecode information; controlling the medical data query monitoring terminal to monitor the retrieval performance of the historical uric acid retrieval information based on asynchronous retrieval operation information, wherein the asynchronous retrieval operation information corresponds to the uric acid retrieval association information; controlling the medical data query terminal to query the uric acid detail retrieval information corresponding to the target user; and A historical uric acid detection dataset of information is retrieved; according to a pre-trained uric acid detection image enhancement model, the initial uric acid detection image is enhanced to generate an enhanced uric acid detection image; uric acid region recognition is performed on the enhanced uric acid detection image to obtain a uric acid recognition region image group; each uric acid recognition region image in the uric acid recognition region image group is optimized to generate an optimized uric acid recognition region image to obtain an optimized uric acid recognition region image group; the historical uric acid detection dataset and the optimized uric acid recognition region image group are input into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

[0102] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the above method of the present disclosure.

[0103] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., provided on the computer device.

[0104] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0105] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A uric acid data collection and abnormality detection method based on multimodal fusion, characterized in that: include: Controlling the uric acid testing device to collect the initial uric acid testing image of the target user; Generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the target user, and generate uric acid retrieval related information based on the retrieval bytecode information; Controlling the medical data query monitoring terminal to monitor the retrieval performance of the historical uric acid retrieval information according to the asynchronous retrieval operation information, wherein the asynchronous retrieval operation information corresponds to the uric acid retrieval related information; Controlling the medical data query terminal to query a historical uric acid test data set corresponding to the uric acid detailed information retrieval information; performing image enhancement processing on the initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; performing uric acid region recognition on the enhanced uric acid detection image to obtain a uric acid recognition region image group; optimizing each uric acid recognition region image in the uric acid recognition region image group to generate an optimized uric acid recognition region image, thereby obtaining an optimized uric acid recognition region image group; The historical uric acid detection dataset and the optimized uric acid recognition area image group are input into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

2. The method according to claim 1, characterized in that The performing image enhancement processing on the initial uric acid detection image according to the pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image includes: Performing surface feature extraction on the initial uric acid detection image using a surface image feature extraction network included in the uric acid detection image enhancement model to generate surface uric acid detection image features; The surface uric acid detection image features are feature-encoded using a multi-level image feature encoding network included in the uric acid detection image enhancement model to generate uric acid detection coded image features; The image enhancement network included in the uric acid detection image enhancement model is used to enhance the uric acid detection encoding image features to generate an enhanced uric acid detection image.

3. The method according to claim 1, characterized in that Optimizing each uric acid recognition area image in the uric acid recognition area image group to generate an optimized uric acid recognition area image includes: Optimizing the sub-region boundaries of the uric acid recognition region image through morphological operations to obtain a first optimized uric acid recognition region image; Topological repair is performed on the first optimized uric acid recognition area image to obtain a repaired uric acid recognition area image as the optimized uric acid recognition area image.

4. The method according to claim 1, wherein The generating corresponding uric acid detailed information retrieval information according to the historical uric acid retrieval information corresponding to the target user includes: Determining, based on the historical uric acid search information, search symptom logic information corresponding to the historical uric acid search information; generating, based on the retrieval disease logic information, combined search formula statement information corresponding to the historical uric acid search information; generating, according to the combined search formula information, call search formula information corresponding to the historical uric acid search information; The retrieval disease logic information, the combined retrieval formula information, and the calling retrieval formula information are combined into uric acid detail retrieval information corresponding to the historical uric acid retrieval information.

5. A uric acid data acquisition and abnormality detection device based on multimodal fusion, characterized in that: include: an acquisition unit configured to control the uric acid detection device to acquire an initial uric acid detection image of a target user; a generating unit configured to generate corresponding uric acid detail retrieval information based on the historical uric acid retrieval information corresponding to the target user, and to generate uric acid retrieval related information based on the retrieval bytecode information; a control unit configured to control the medical data query monitoring terminal to monitor the retrieval performance of the historical uric acid retrieval information according to asynchronous retrieval operation information, wherein the asynchronous retrieval operation information corresponds to the uric acid retrieval related information; a query unit configured to control the medical data query terminal to query a historical uric acid test data set corresponding to the uric acid detailed information retrieval information; an enhancement unit configured to perform image enhancement processing on the initial uric acid detection image according to a pre-trained uric acid detection image enhancement model to generate an enhanced uric acid detection image; an identification unit configured to perform uric acid region identification on the enhanced uric acid detection image to obtain a uric acid identification region image group; an optimization unit configured to optimize each uric acid recognition region image in the uric acid recognition region image group to generate an optimized uric acid recognition region image, thereby obtaining an optimized uric acid recognition region image group; The input unit is configured to input the historical uric acid detection data set and the optimized uric acid recognition area image group into a pre-trained multimodal uric acid abnormality detection model to generate a uric acid abnormality detection result.

6. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable medium, characterized in that A computer program is stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images