Mobile ultrasonic cloud diagnosis method and device, computer equipment and storage medium

Abstract

The invention discloses a mobile ultrasonic cloud diagnosis method. The method comprises the following steps: S1, acquiring ultrasonic image data of a patient; s2, preprocessing the ultrasonic image data, and extracting multi-level feature representation; s3, classifying and identifying the multi-level feature representation, and identifying a lesion or a focus; and S4, generating a diagnosis suggestion based on the identified lesion or focus. The invention further provides a mobile ultrasonic cloud diagnosis device, computer equipment and a storage medium. According to the method, the ultrasonic image can be finely analyzed, the accuracy of feature extraction and recognition is improved, the uploaded image can be processed in real time, the waiting time of a user is shortened, automation of the whole process is achieved from image preprocessing to diagnosis suggestion generation, and the burden of a doctor is greatly relieved.

Description

Technical Field

[0001] The present invention relates to the field of ultrasound technology, and more specifically, to a mobile ultrasound cloud diagnosis method, device, computer equipment and storage medium. Background Art

[0002] With the development of mobile Internet and the advancement of medical technology, mobile health applications have gradually become an important way of modern medical services. Ultrasound diagnosis, as a non-invasive, convenient and effective medical method, is widely used in medical diagnosis. However, traditional ultrasound diagnosis often relies on large equipment and professional technicians, and patients need to go to medical institutions in person for examination, which is significantly inconvenient in remote areas and areas with scarce medical resources. Existing technologies lack convenient and real-time remote ultrasound diagnosis methods, which makes it difficult to meet user needs. Summary of the invention

[0003] The purpose of the present invention is to provide a mobile ultrasonic cloud diagnosis method to overcome the defects of the prior art.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A mobile ultrasound cloud diagnosis method comprises the following steps:

[0006] S1. Collecting ultrasound image data of the patient;

[0007] S2, preprocessing the ultrasound image data to extract multi-level feature representation;

[0008] S3, classifying and identifying the multi-level feature representation to identify lesions or lesions;

[0009] S4. Generate diagnostic recommendations based on the identified lesions or lesions.

[0010] Furthermore, in step S1, a portable ultrasound device is used to collect ultrasound images of the patient.

[0011] Furthermore, the step S2 specifically includes:

[0012] Standardize the size of ultrasound images, grayscale them, enhance them, and remove noise;

[0013] The convolutional neural network is used to extract features of different levels from the preprocessed ultrasound images to obtain a multi-level feature representation.

[0014] Furthermore, the convolutional neural network is used to extract features of different levels from the preprocessed ultrasound image to obtain the steps of multi-level feature representation:

[0015] The ultrasound image is convolved by a convolutional neural network to extract a multi-level feature representation, which includes:

[0016] Shallow features describe the basic elements of an image;

[0017] Mid-level features describe specific shapes or local structures in the image;

[0018] High-level features describe the overall layout of the image, scene classification, and features related to diagnostic criteria such as lesion type, lesion size, and location.

[0019] Furthermore, the step S3 specifically includes:

[0020] The Transformer model used was pre-trained on multiple public medical ultrasound image datasets. During the training process, the model's ability to identify lesion features in these datasets was optimized. The model architecture took into account the characteristics of ultrasound images, including resolution, contrast, and texture information, to ensure that key visual features can be effectively extracted for analysis and identification of lesions or lesions. The specific steps are as follows:

[0021] Data preparation: collecting and annotating a set amount of ultrasound image data, wherein the ultrasound image data includes normal and abnormal images, and the corresponding lesion locations and types;

[0022] Data preprocessing: Perform size standardization, grayscale conversion, image enhancement, and noise removal on ultrasound image data to improve the training efficiency and performance of the model;

[0023] Feature extraction: Use convolutional neural networks to extract multi-level feature representations from preprocessed images, including shallow features, mid-level features, and high-level features;

[0024] Model architecture design: Design the architecture of the Transformer model, including self-attention layer, feedforward neural network layer, and layer normalization components;

[0025] The model training process includes:

[0026] First, the extracted features are input into the Transformer model;

[0027] Secondly, use the loss function to measure the difference between the model's prediction and the true annotation;

[0028] Then, the weights of the model are updated through the back-propagation algorithm to minimize the loss function;

[0029] Finally, the model is evaluated using the validation set to avoid overfitting and perform hyperparameter tuning.

[0030] Model optimization: During the training process, you may need to tune the model, including adjusting the learning rate, batch size, and regularization strategy;

[0031] Model evaluation: Evaluate the performance of the model on an independent test set, using accuracy, recall, and F1 score metrics;

[0032] Model deployment: The trained model can be deployed to the mobile ultrasound cloud diagnosis system to analyze ultrasound images in real time and provide diagnostic recommendations.

[0033] Furthermore, the step S4 specifically includes:

[0034] Based on the identified lesions or lesions and in combination with medical guidelines and expert systems, the patient's age, gender, medical history, and the nature, location, and size of the lesions are taken into consideration, and the latest clinical research results are combined to provide the most accurate diagnostic recommendations.

[0035] Furthermore, after step S4, the following steps are further included:

[0036] Provide diagnostic recommendations to patients via mobile means.

[0037] In order to solve the above technical problems, the present application also provides a mobile ultrasonic cloud diagnostic device, which adopts the following technical solution:

[0038] A mobile ultrasonic cloud diagnostic device, comprising:

[0039] A portable ultrasound device module for collecting ultrasound image data of a patient;

[0040] An artificial intelligence processing module, used to pre-process the ultrasound image data and extract multi-level feature representation;

[0041] A classification and recognition module, used to classify and recognize the multi-level feature representation to identify lesions or lesions;

[0042] A suggestion generation module is used to generate diagnostic suggestions based on the identified lesions or lesions.

[0043] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0044] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the mobile ultrasound cloud diagnosis method as described above when executing the computer-readable instructions.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0046] A computer-readable storage medium, characterized in that computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the steps of the mobile ultrasound cloud diagnosis method as described above are implemented.

[0047] Compared with the prior art, the advantages of the present invention are: the present invention can perform fine analysis of ultrasound images, improve the accuracy of feature extraction and recognition, can process uploaded images in real time, shorten user waiting time, and realize automation of the entire process from image preprocessing to diagnosis suggestion generation, greatly reducing the burden on doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 It is a flow chart of the mobile ultrasonic cloud diagnosis method of the present invention.

[0050] Figure 2 It is a framework diagram of the ultrasonic remote diagnosis device of the present invention.

[0051] Figure 3 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0053] When the present invention is applied specifically, it includes a portable ultrasound device, a mobile application, and a cloud medical service platform, wherein the portable ultrasound device is used to collect ultrasound image data, the mobile application can be installed on the user's mobile terminal, and the cloud medical service platform is installed as a server in a medical institution. The mobile ultrasound cloud diagnosis method of this embodiment runs on the mobile application to realize data flow and processing between the portable ultrasound device and the cloud medical service platform.

[0054] See also Figure 1 As shown, this embodiment discloses a mobile ultrasound cloud diagnosis method, comprising the following steps:

[0055] Step S1: Acquire ultrasonic image data of a patient.

[0056] In this embodiment, the user registers and logs in through the mobile application, enters the cloud diagnosis platform, and then collects ultrasound image data by connecting to a portable ultrasound device and uploads it to the cloud.

[0057] Step S2: preprocess the ultrasound image data to extract multi-level feature representation.

[0058] In this embodiment, the preprocessing mainly adjusts the ultrasound image to a format that can be effectively processed by the model and optimizes the image quality. Specifically, step S2 specifically includes:

[0059] The ultrasound image is processed by image size standardization, grayscale, image enhancement (contrast, brightness adjustment) and noise removal, which can make the image more suitable for the input requirements of the model and retain key features.

[0060] The convolutional neural network (CNN model) is used to extract features of different levels from the preprocessed ultrasound image to obtain a multi-level feature representation. The multi-level features include:

[0061] Shallow features: usually include basic elements of the image, such as edges, colors, and textures. These are the most intuitive and basic information of the image and are the cornerstone for building more advanced features.

[0062] Mid-level features: Based on shallow-level features, mid-level features can describe certain specific patterns in the image, such as specific shapes or local structures. These features are more complex than shallow-level features and can express richer visual information.

[0063] High-level features: Features that describe the overall layout of the image, scene classification, and diagnostic conditions such as lesion type, lesion size, and location. The deepest features are global and abstract representations of the entire image or the main objects in the image. These features involve the overall layout of the image, scene classification, and features related to specific diagnostic conditions.

[0064] Through such multi-level feature representation, deep learning models can understand image content from different perspectives and levels, providing support for accurate diagnosis and identification.

[0065] Step S3: classify and identify the multi-level feature representation to identify the lesions or lesions.

[0066] In this embodiment, a pre-trained Transformer model is used to analyze the multi-level feature representation to identify lesions or lesions; the Transformer model can process the associations between different areas in the image through the self-attention mechanism, thereby capturing more complex patterns and relationships, which is crucial for classifying specific medical image features.

[0067] Specifically, the classification layer of the Transformer model (commonly found in the last few layers of the model) receives the encoded features from the previous step and learns to distinguish different types of lesions or health conditions through training. For example, through techniques such as logistic regression and softmax function, the model is able to choose between multiple possible disease categories.

[0068] The Transformer model used was pre-trained on multiple public medical ultrasound image datasets. During the training process, the model's ability to identify lesion features in these datasets was optimized. The model architecture took into account the characteristics of ultrasound images, including resolution, contrast, and texture information, to ensure that key visual features can be effectively extracted for analysis and identification of lesions or lesions. The specific steps are as follows:

[0069] Data preparation: collecting and annotating a set amount of ultrasound image data, wherein the ultrasound image data includes normal and abnormal images, and the corresponding lesion locations and types;

[0070] Data preprocessing: Perform size standardization, grayscale conversion, image enhancement, and noise removal on ultrasound image data to improve the training efficiency and performance of the model;

[0071] Feature extraction: Use convolutional neural networks to extract multi-level feature representations from preprocessed images, including shallow features, mid-level features, and high-level features;

[0072] Model architecture design: Design the architecture of the Transformer model, including self-attention layer, feedforward neural network layer, and layer normalization components;

[0073] The model training process includes:

[0074] First, the extracted features are input into the Transformer model;

[0075] Secondly, use the loss function to measure the difference between the model's prediction and the true annotation;

[0076] Then, the weights of the model are updated through the back-propagation algorithm to minimize the loss function;

[0077] Finally, the model is evaluated using the validation set to avoid overfitting and perform hyperparameter tuning.

[0078] Model optimization: During the training process, you may need to tune the model, including adjusting the learning rate, batch size, and regularization strategy;

[0079] Model evaluation: Evaluate the performance of the model on an independent test set, using accuracy, recall, and F1 score metrics;

[0080] Model deployment: The trained model can be deployed to the mobile ultrasound cloud diagnosis system to analyze ultrasound images in real time and provide diagnostic recommendations.

[0081] Step S4: Generate diagnostic suggestions based on the identified lesions or lesions.

[0082] In this embodiment, doctors can also be notified through the cloud diagnosis platform to conduct real-time consultations. Doctors can view images and diagnostic suggestions through the platform and interact with users. Doctors can generate diagnostic reports based on processed image data and provide feedback to users through mobile applications.

[0083] The present invention can realize real-time and convenient ultrasonic remote diagnosis services, effectively solve the shortcomings of the existing technology, can finely analyze ultrasonic images, improve the accuracy of feature extraction and recognition, can process uploaded images in real time, shorten user waiting time, and realize the automation of the entire process from image preprocessing to diagnosis suggestion generation, greatly reducing the burden on doctors.

[0084] Based on the identified lesions or lesions and in combination with medical guidelines and expert systems, the patient's age, gender, medical history, and the nature, location, and size of the lesions are taken into consideration, and the latest clinical research results are combined to provide the most accurate diagnostic recommendations.

[0085] See also Figure 2 As shown, in order to solve the above technical problems, the embodiment of the present application also provides an ultrasonic remote diagnosis device, which adopts the following technical solution: a mobile ultrasonic cloud diagnosis device, comprising:

[0086] The portable ultrasound device module is used to collect ultrasound image data of the patient, and upload the collected data to the cloud server module for storage through the network communication module;

[0087] An artificial intelligence processing module is used to preprocess the ultrasound image data and extract multi-level feature representations; a classification and recognition module is used to classify and recognize the multi-level feature representations and identify lesions or foci; and a suggestion generation module is used to generate diagnostic suggestions based on the identified lesions or foci.

[0088] The mobile ultrasonic cloud diagnostic device of the present invention also includes: a doctor consultation module, which is used for doctors to view images and diagnostic suggestions in real time and interact with users, and a user feedback module, which is used to feed back diagnostic reports to user terminals.

[0089] The present invention can perform fine-grained analysis of ultrasound images, improve the accuracy of feature extraction and recognition, process uploaded images in real time, shorten user waiting time, and automate the entire process from image preprocessing to diagnostic suggestion generation, greatly reducing the burden on doctors.

[0090] To solve the above technical problems, the present application also provides a computer device. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0091] The computer device 3 includes a memory 31, a processor 32, and a network interface 33 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 3 with components 31-33, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0092] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0093] The memory 31 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 31 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 31 can also be an external storage device of the computer device 7, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 7. Of course, the memory 31 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 31 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions of the mobile ultrasound cloud diagnosis method, etc. In addition, the memory 31 can also be used to temporarily store various types of data that have been output or are to be output.

[0094] The processor 32 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 32 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 32 is used to run computer-readable instructions stored in the memory 31 or process data, such as computer-readable instructions for running the mobile ultrasound cloud diagnosis method.

[0095] The network interface 33 may include a wireless network interface or a wired network interface. The network interface 33 is generally used to establish a communication connection between the computer device 3 and other electronic devices.

[0096] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the mobile ultrasound cloud diagnosis method as described above.

[0097] By combining ultrasonic diagnostic equipment with a telemedicine platform and artificial intelligence, the present invention not only improves the accuracy and efficiency of diagnosis, but also can provide timely medical services in an environment with scarce medical resources, thereby improving patient satisfaction and treatment effects.

[0098] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, the patent owner may make various variations or modifications within the scope of the appended claims. As long as they do not exceed the protection scope described in the claims of the present invention, they should be within the protection scope of the present invention.

Claims

1. A mobile ultrasonic cloud diagnosis method, characterized in that: The following steps are involved: S1, collecting ultrasound image data of the patient; S2, preprocessing the ultrasound image data to extract multi-level feature representation; S3, classifying and identifying the multi-level feature representation to identify lesions or lesions; S4. Generate diagnostic recommendations based on the identified lesions or lesions.

2. The mobile ultrasonic cloud diagnosis method according to claim 1, characterized in that: In step S1, a portable ultrasound device is used to collect ultrasound images of the patient.

3. The mobile ultrasonic cloud diagnosis method according to claim 1, characterized in that: The step S2 specifically includes: Standardize the size of ultrasound images, grayscale them, enhance them, and remove noise; The convolutional neural network is used to extract features of different levels from the preprocessed ultrasound images to obtain a multi-level feature representation.

4. The mobile ultrasonic cloud diagnosis method according to claim 3, characterized in that: The steps of extracting different levels of features from the preprocessed ultrasound image through a convolutional neural network and obtaining a multi-level feature representation are as follows: The ultrasound image is convolved by a convolutional neural network to extract a multi-level feature representation, which includes: Shallow features describe the basic elements of an image; Mid-level features describe specific shapes or local structures in the image; High-level features describe the overall layout of the image, scene classification, and features related to diagnostic criteria such as lesion type, lesion size, and location.

5. The mobile ultrasonic cloud diagnosis method according to claim 1, characterized in that: The step S3 specifically includes: The Transformer model used was pre-trained on multiple public medical ultrasound image datasets. During the training process, the model's ability to identify lesion features in these datasets was optimized. The model architecture took into account the characteristics of ultrasound images, including resolution, contrast, and texture information, to ensure that key visual features can be effectively extracted for analysis and identification of lesions or lesions. The specific steps are as follows: Data preparation: collecting and annotating a set amount of ultrasound image data, wherein the ultrasound image data includes normal and abnormal images, and the corresponding lesion locations and types; Data preprocessing: Perform size standardization, grayscale conversion, image enhancement, and noise removal on ultrasound image data to improve the training efficiency and performance of the model; Feature extraction: Use convolutional neural networks to extract multi-level feature representations from preprocessed images, including shallow features, mid-level features, and high-level features; Model architecture design: Design the architecture of the Transformer model, including self-attention layer, feedforward neural network layer, and layer normalization components; The model training process includes: First, the extracted features are input into the Transformer model; Secondly, use the loss function to measure the difference between the model's prediction and the true annotation; Then, the weights of the model are updated through the back-propagation algorithm to minimize the loss function; Finally, the model is evaluated using the validation set to avoid overfitting and perform hyperparameter tuning. Model optimization: During the training process, you may need to tune the model, including adjusting the learning rate, batch size, and regularization strategy; Model evaluation: Evaluate the performance of the model on an independent test set, using accuracy, recall, and F1 score metrics; Model deployment: The trained model can be deployed to the mobile ultrasound cloud diagnosis system to analyze ultrasound images in real time and provide diagnostic recommendations.

6. The mobile ultrasonic cloud diagnosis method according to claim 1, characterized in that: The step S4 specifically includes: Based on the identified lesions or lesions and in combination with medical guidelines and expert systems, the patient's age, gender, medical history, and the nature, location, and size of the lesions are taken into consideration, and the latest clinical research results are combined to provide the most accurate diagnostic recommendations.

7. The mobile ultrasonic cloud diagnosis method according to claim 1, characterized in that: After step S4, the following steps are also included: Provide diagnostic recommendations to patients via mobile means.

8. A device for implementing the mobile ultrasonic cloud diagnosis method according to any one of claims 1 to 7, characterized in that: include: A portable ultrasound device module for collecting ultrasound image data of a patient; An artificial intelligence processing module, used to pre-process the ultrasound image data and extract multi-level feature representation; A classification and recognition module, used to classify and recognize the multi-level feature representation to identify lesions or lesions; A suggestion generation module is used to generate diagnostic suggestions based on the identified lesions or lesions.

9. A computer device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the mobile ultrasound cloud diagnosis method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the mobile ultrasound cloud diagnosis method according to any one of claims 1 to 7 are implemented.

Images