Online inquiry method and device capable of sending graph, equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and medical treatment, and relates to an online inquiry method capable of sending pictures, which comprises the following steps: acquiring a reception log set of a real doctor, screening out pictures sent to a patient by the doctor, and obtaining an initial picture set; performing clustering processing on the initial picture set to obtain a middle picture set, and selecting a preset number of pictures from each cluster according to the picture use frequency to obtain a target picture set; performing text mapping on the target picture set to obtain text description of each target picture, and taking the text description and the corresponding target picture as training data; training the initial inquiry model by using the training data to obtain a target inquiry model; and acquiring inquiry information sent by the patient, identifying the inquiry information by using the target inquiry model to obtain picture information and a binding text corresponding to the inquiry information, and feeding back the picture information and the binding text corresponding to the inquiry information to the patient. In addition, the invention also relates to a block chain technology, and inquiry information and the like can be stored in a block chain. According to the invention, the accuracy of online inquiry can be improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and medical treatment, and in particular to an online medical consultation method capable of sending pictures. Background Art

[0002] In today's society, with the accelerated pace of life and the growing demand for medical care, offline consultations face long waiting times and inconvenient medical care. In this context, the rapid development of Internet technology has provided a new solution for medical services. For minor illnesses such as colds, minor injuries, and abdominal pain, patients are increasingly inclined to obtain medical services through online consultation systems, which has become an important way for people to obtain medical consultations and preliminary diagnoses.

[0003] However, in the current online consultation process, the communication between patients and doctors mainly relies on text interaction. This single communication method has many limitations, especially when describing their own conditions. Due to the lack of professional medical knowledge or limited expression ability of patients, it is often difficult to accurately describe the pain location or related symptoms, which seriously affects the accuracy of online consultation. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an online medical consultation method, device, equipment and storage medium that can publish pictures, and its main purpose is to improve the accuracy of online medical consultation.

[0005] In order to solve the above technical problems, the embodiment of the present application provides an online consultation method capable of sending pictures, which adopts the following technical solutions:

[0006] Obtaining a set of consultation logs of real doctors from a preset consultation database, and filtering out pictures sent by doctors to patients from the consultation logs to obtain an initial set of pictures;

[0007] Performing clustering processing on the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes a plurality of clusters;

[0008] Based on each cluster of the intermediate picture set, a preset number of pictures are selected from each cluster according to the frequency of use of the pictures to obtain a target picture set;

[0009] Performing text mapping on each target image in the target image set to obtain a text description of each target image;

[0010] Collect all target images with text descriptions to obtain training data, and use the training data to train the initial consultation model to obtain the target consultation model;

[0011] When an online consultation is received, the consultation information sent by the patient is obtained, the consultation information is identified using the target consultation model, the image information and bound text corresponding to the consultation information are obtained, and the image information and bound text corresponding to the consultation information are fed back to the patient.

[0012] Furthermore, clustering the initial picture set to obtain an intermediate picture set includes:

[0013] Performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture;

[0014] Pre-building a fast similarity vector retrieval library and an index, storing the initial image with a hash value in the fast similarity vector retrieval library, traversing the fast similarity vector retrieval library, and calculating the similarity value of each initial image in the fast similarity vector retrieval library through the index;

[0015] The fast similarity vector retrieval library is clustered according to the similarity value of each of the initial pictures using a preset clustering algorithm to obtain an intermediate picture set including a plurality of clusters.

[0016] Furthermore, performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture includes:

[0017] Performing discrete cosine transformation on each initial picture in the initial picture set to obtain an image transformation result;

[0018] Extracting low-frequency coefficients from the image transformation result, and calculating the mean of the low-frequency coefficients to generate a binary image;

[0019] The binary images are formatted to obtain a hash value corresponding to each of the initial images.

[0020] Furthermore, performing text mapping on each target image in the target image set to obtain a text description of each target image includes:

[0021] Using a pre-trained feature extraction model to extract features from each target image in the target image set to obtain target image features;

[0022] Inputting the target image features into a pre-trained classification model, processing the target image features through multiple sub-models of the classification model, and obtaining recognition results of the multiple sub-models;

[0023] Generate binding text of the target image feature according to the recognition results of the plurality of sub-models and a preset fixed template;

[0024] The recognition results of the multiple sub-models and the bound text are aggregated to obtain a text description of each target image.

[0025] Furthermore, after obtaining the text description of each target image, the method further includes:

[0026] Sending the target image and the text description of the target image to the review end for review;

[0027] If the review fails, the target image that fails the review and the text description corresponding to the target image will be deleted.

[0028] Furthermore, when receiving the online consultation, after obtaining the consultation information sent by the patient, the method further includes:

[0029] Identify the data type of the medical inquiry information;

[0030] If the data type is identified as a non-text form, a text conversion method corresponding to the non-text form is obtained;

[0031] The medical consultation information is converted into a text form according to a text conversion method corresponding to the non-text form.

[0032] Furthermore, the target medical inquiry model is used to identify the medical inquiry information, obtain the picture information and the bound text corresponding to the medical inquiry information, and feed back the picture information and the bound text corresponding to the medical inquiry information to the patient, including:

[0033] Receiving medical inquiry information through the input module of the target medical inquiry model, and extracting text features of the medical inquiry information and image features of each compliant image in the compliant image library through the feature extraction module of the target medical inquiry model;

[0034] Calculating a matching value between the text feature and each of the image features through a matching module of the target medical inquiry model, and determining a compliance image corresponding to the medical inquiry information from the compliance image library according to the matching value;

[0035] Extracting the binding text of the compliance image by using the binding text extraction module of the medical inquiry model;

[0036] The compliance image and the bound text of the compliance image are fed back to the patient through the output module of the medical inquiry model.

[0037] In order to solve the above technical problems, the embodiment of the present application further provides an online medical consultation device capable of sending pictures, which adopts the following technical solution:

[0038] An acquisition module is used to acquire a set of consultation logs of real doctors from a preset consultation database, and to filter out pictures sent by doctors to patients from the consultation logs to obtain an initial set of pictures;

[0039] A clustering module, configured to perform clustering processing on the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes a plurality of clusters;

[0040] A screening module, configured to select a preset number of pictures from each cluster of the intermediate picture set according to the frequency of picture usage, to obtain a target picture set;

[0041] A mapping module, used to perform text mapping on each target image in the target image set to obtain a text description of each target image;

[0042] A training module is used to collect all target images with text descriptions to obtain training data, and use the training data to train the initial consultation model to obtain a target consultation model;

[0043] The feedback module is used to obtain the inquiry information sent by the patient when receiving an online consultation, identify the inquiry information using the target inquiry model, obtain the picture information and bound text corresponding to the inquiry information, and feed back the picture information and bound text corresponding to the inquiry information to the patient.

[0044] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the online consultation method that can send images as described above.

[0045] In order to solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the online medical consultation method capable of sending images as described above.

[0046] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0047] In this embodiment, the pictures sent by the doctors to the patients are screened based on the medical records of the real doctors and the patients to obtain the initial picture set. Since only the pictures sent by the real doctors to the patients are screened, the compliance of the initial picture set can be guaranteed, thereby improving the accuracy of subsequent online consultations and avoiding the inclusion of low-quality pictures into the initial picture set, which would cause inaccurate subsequent model training or require additional picture compliance verification steps.

[0048] By clustering the initial image set using a preset clustering method, similar images can be grouped together, reducing data redundancy and providing more accurate training data for subsequent consultation model training, thereby improving the accuracy of subsequent consultation models.

[0049] According to the frequency of use of the pictures, a preset number of pictures are selected in each cluster to ensure that the pictures in the target picture set are highly representative and practical, which can improve the accuracy and generalization ability of the model, thereby improving the accuracy of online consultation; in the subsequent consultation model training steps, the consumption of computing resources and the occupation of storage space are reduced, thereby improving the efficiency of model training;

[0050] By performing text mapping on each target image in the target image set and obtaining a text description of each target image, the model can more accurately understand the image content and context, thereby improving training efficiency and accuracy; by not showing the user more detailed information than necessary, information overload and interference can be reduced, allowing the user to focus more on the image text and bound text, thereby improving the accuracy and efficiency of user decision-making;

[0051] By collecting all target images with text descriptions to obtain training data, the initial consultation model is trained using the training data to obtain a target consultation model. By using the screened target images with text descriptions as training data, the accuracy and compliance of the subsequent target consultation model in processing data can be improved, the accuracy and efficiency of the online consultation model can be improved, and the user experience can be improved.

[0052] The target medical inquiry model is used to identify the medical inquiry information, and the image information and bound text corresponding to the medical inquiry information are obtained, and the image information and the bound text corresponding to the medical inquiry information are fed back to the patient, which can automatically guide the patient to indicate the exact pain site. The specific pain site of the patient can be known without the guidance of the doctor, thereby improving the efficiency and accuracy of online medical consultations; guidance is provided through clear and concise pictures and bound texts, which prevents doctors from misdiagnosing due to unclear patient expressions, thereby improving the accuracy of online medical consultations, and this method can reduce the workload of doctors, thereby being able to efficiently handle more online medical consultation requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0055] Figure 2 A flow chart of an embodiment of an online medical consultation method that can publish images according to the present application;

[0056] Figure 3 It is a structural schematic diagram of an embodiment of an online medical consultation device capable of publishing images according to the present application;

[0057] Figure 4 It is a structural schematic diagram of an embodiment of a device according to the present application. DETAILED DESCRIPTION

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0059] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0061] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0062] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0063] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0064] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0065] It should be noted that the online medical consultation method capable of sending pictures provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the online medical consultation device capable of sending pictures is generally arranged in a server / terminal device.

[0066] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0067] Continue to refer Figure 2 , showing a flowchart of an embodiment of an online consultation method with pictures that can be sent according to the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The online consultation method with pictures that can be sent provided in the embodiment of the present application can be applied to any scenario that requires an online consultation with pictures, and the online consultation method with pictures that can be sent can be applied to products in these scenarios. The online consultation method with pictures that can be sent includes the following steps:

[0068] Step S201: Obtain a consultation log set of a real doctor from a preset consultation database, and filter out pictures sent by the doctor to the patient from the consultation log set to obtain an initial picture set.

[0069] In this embodiment, the online consultation method capable of sending pictures is executed on an electronic device (eg Figure 1The server / terminal device shown in the figure) can preset the initial picture set of real doctors in the consultation database through a wired connection or a wireless connection. It should be pointed out that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future. The executor of the present application may be a target consultation model, or simply referred to as a model. The above-mentioned initial picture set is usually the same size as the picture when the target consultation model is sent, but the training data contains other noise, depending on the specific situation. Among them, in the business scenario of online consultation that can be sent, the initial picture set is obtained from the preset database by screening the consultation log set of real doctors in the preset consultation database.

[0070] In this embodiment, the above-mentioned preset consultation database stores a set of consultation logs of all real doctors when they see patients, wherein the consultation log set includes pictures sent by patients to real doctors, pictures sent by real doctors to patients, and interactive texts between real doctors and patients, etc.; pictures sent by real doctors to patients are filtered out from all consultation log sets to obtain an initial picture set, for example, the initial picture set includes pathology (such as images of eczema, images of psoriasis, etc.), abdominal images (such as indicating the location of the abdomen, right upper abdomen, upper abdomen, left upper abdomen, etc.), knee images, ankle images, etc.

[0071] In this embodiment, the initial picture set is obtained by screening the pictures sent by the doctors to the patients based on the medical records of the real doctors and the patients. Since only the pictures sent by the real doctors to the patients are screened, the compliance of the initial picture set can be guaranteed, thereby improving the accuracy of subsequent online consultations and avoiding the inclusion of low-quality pictures in the initial picture set, which would cause inaccurate subsequent model training or require a new picture compliance verification step.

[0072] Step S202: clustering the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes a plurality of clusters.

[0073] In this embodiment, a preset clustering method is used to cluster the initial picture set, wherein the preset clustering algorithm includes but is not limited to K-Means clustering, DBSCAN clustering, mean shift clustering, GMM clustering, agglomerative hierarchical clustering, etc.; clustering parameters are preset according to the size and complexity of the initial picture set, and the initial picture set is clustered according to the clustering method and the corresponding clustering parameters. When the clustering is completed, many clusters are generated, and an intermediate picture set containing multiple clusters is obtained, wherein each cluster has a unique identifier, and the pictures in each cluster are similar pictures, and the size of the cluster is represented by the picture data contained in the cluster.

[0074] In this embodiment, the initial image set is clustered using a preset clustering method, which can classify similar images into one category, reducing data redundancy and providing more accurate training data for the training of subsequent consultation models, thereby improving the accuracy of subsequent consultation models.

[0075] Step S203 : Based on each cluster of the intermediate picture set, a preset number of pictures are selected from each cluster according to the frequency of picture usage to obtain a target picture set.

[0076] In this embodiment, the goal is to select a preset number of pictures from each cluster based on the intermediate picture set, for example, select the top 10 pictures with the highest usage frequency in each cluster, to construct a target picture set.

[0077] Specifically, the consultation log set is traced back, the number of times each picture in the intermediate picture set was sent by the doctor is counted, and the statistical results of each picture are obtained. The statistical results can be a list representation including the picture ID and the corresponding number of times used. The pictures in each cluster are sorted according to the frequency of use to obtain multiple clusters with a sorting order. According to a preset number, the top N pictures (N is a preset number) are selected from each sorted cluster as representative pictures of the cluster. When the number of pictures in a cluster is less than the preset number, all pictures of the cluster can be retained. Finally, all the filtered pictures are merged to obtain the target picture set.

[0078] In this embodiment, a preset number of pictures are selected in each cluster based on the frequency of use of the pictures to ensure that the pictures in the target picture set are highly representative and practical, which can improve the accuracy and generalization ability of the model, thereby improving the accuracy of online consultations; in the subsequent consultation model training steps, the consumption of computing resources and the occupancy of storage space are reduced, thereby improving the efficiency of model training.

[0079] Step S204: performing text mapping on each target image in the target image set to obtain a text description of each target image;

[0080] In this embodiment, each target image in the target image set is extracted, and text mapping is performed on each image to obtain a text description of each target image, wherein the contents of the text mapping include department, image type, image subcategory, image text, bound text, etc.; wherein, department refers to the medical department to which the identified image belongs, such as obstetrics and gynecology, pediatrics, surgery, etc.; image type is a basic type for describing images, such as part, instrument, pathology, etc.; image subcategory is further subdivided on the basis of image type, such as abdomen, head, X-ray, CT scan, etc.; image text refers to the text description or label in the image, such as the label indicating each part of the abdomen in the abdominal image: 1-right upper abdomen, 2-upper abdomen, 3-left upper abdomen, 4-right waist, 5-middle abdomen, 6-left waist, 7-right lower abdomen, 8-lower abdomen, 9-left lower abdomen; bound text is a text description or guide related to the image, such as in the above-mentioned abdominal image, the bound text is "Please indicate the specific pain site according to the abdominal image."

[0081] It is worth mentioning that the image type, image subcategory, and department mapped by the above text can be set to be used for model training. They do not need to be displayed to users in the trained consultation model. Only the image text and bound text need to be displayed.

[0082] In this embodiment, by performing text mapping on each target image in the target image set to obtain a text description of each target image, the model can more accurately understand the image content and context, thereby improving training efficiency and accuracy; by not showing the user more detailed information than necessary, information overload and interference can be reduced, allowing the user to focus more on the image text and bound text, thereby improving the accuracy and efficiency of the user's decision-making.

[0083] Step S205: collect all target images with text descriptions to obtain training data, and use the training data to train the initial medical inquiry model to obtain the target medical inquiry model.

[0084] In this embodiment, all target images with text descriptions are collected, and text data of a large number of patients' online consultations are collected, and the text data of online consultations and target images with text descriptions are used as training data; an initial consultation model is pre-constructed, and the initial consultation model is trained using the training data to finally obtain a trained target consultation model, wherein the target consultation model has the function of sending pictures to patients, wherein before the initial consultation model is trained using the training data, the training data also needs to be pre-processed, and the pre-processing includes but is not limited to image denoising, text segmentation, normalization, etc.

[0085] The above-mentioned initial consultation model includes an input module, a feature extraction module, a matching module, a bound text extraction module and an output module; the initial consultation model is trained by the above-mentioned training data, and the matching degree between the text features and the image features is calculated by the loss function, wherein the loss function includes but is not limited to cosine similarity loss function, cross entropy loss function, etc. The cosine similarity loss function measures the similarity by calculating the cosine value between the feature vectors, and the larger the value, the more similar it is; the cross entropy loss function is often used for classification problems, and guides model optimization by calculating the difference between the predicted probability and the actual label.

[0086] In this embodiment, training data is obtained by collecting all target images with text descriptions, and the initial consultation model is trained using the training data to obtain a target consultation model. By using the screened target images with text descriptions as training data, the accuracy and compliance of data processed by the subsequent target consultation model can be improved, the accuracy and efficiency of the online consultation model can be improved, and the user experience can be improved.

[0087] Step S206: When an online consultation is received, the consultation information sent by the patient is obtained, the consultation information is identified using the target consultation model, the image information and bound text corresponding to the consultation information are obtained, and the image information and bound text corresponding to the consultation information are fed back to the patient.

[0088] In this embodiment, when an online outpatient process initiated by a patient is received, the medical inquiry information submitted by the patient is obtained, and the medical inquiry information is preprocessed. The preprocessing operation includes removing useless characters, word segmentation operations, etc. The preprocessed medical inquiry information is subjected to feature extraction operations through the trained target medical inquiry model to obtain text features. The text features are matched with a compliance database storing compliance images to obtain image information corresponding to the medical inquiry information. The image information and the bound text corresponding to the image information are fed back to the user. For example, the received medical inquiry information is "My stomach hurts, what is the problem?" The medical inquiry information is input into the target medical inquiry model. A feature extraction operation is performed on the sentence "My stomach hurts, what's the problem?", and the text features "stomach" and "pain" are extracted; the text features are matched with the corresponding abdominal area image in the compliance database, and the abdominal area image has the following labels: 1-right upper abdomen, 2-upper abdomen, 3-left upper abdomen, 4-right waist, 5-middle abdomen, 6-left waist, 7-right lower abdomen, 8-lower abdomen, 9-left lower abdomen; and the bound text of the abdominal area image "Please indicate the specific pain area according to the abdominal image" is fed back to the patient. After the patient indicates the specific area of ​​pain, the doctor who receives the patient will know the more specific situation and provide symptomatic medication or treatment suggestions.

[0089] In this embodiment, the target medical inquiry model is used to identify the medical inquiry information, and the image information and bound text corresponding to the medical inquiry information are obtained, and the image information and the bound text corresponding to the medical inquiry information are fed back to the patient, which can automatically guide the patient to indicate the exact pain site. The specific pain site of the patient can be known without the guidance of the doctor, thereby improving the efficiency and accuracy of online medical consultations; guidance through clear and concise pictures and bound texts can prevent doctors from misdiagnosing due to unclear patient expressions, thereby improving the accuracy of online medical consultations, and this method can reduce the workload of doctors, thereby being able to efficiently handle more online medical consultation requests.

[0090] In one embodiment, clustering the initial picture set to obtain an intermediate picture set includes:

[0091] Performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture;

[0092] Pre-building a fast similarity vector retrieval library and an index, storing the initial image with a hash value in the fast similarity vector retrieval library, traversing the fast similarity vector retrieval library, and calculating the similarity value of each initial image in the fast similarity vector retrieval library through the index;

[0093] The fast similarity vector retrieval library is clustered according to the similarity value of each of the initial pictures using a preset clustering algorithm to obtain an intermediate picture set including a plurality of clusters.

[0094] In this embodiment, the preset hash algorithm refers to a perceptual hash algorithm (pHash), which is an image hash technology used to generate a hash value of an image;

[0095] The hash value of each initial image in the initial image set is calculated by the perceptual hash algorithm; and a fast similarity vector retrieval library is constructed by the FAISS tool. The above-mentioned FAISS (Facebook AI Similarity Search) tool is a library for efficiently searching similar vectors in a large number of vector sets. All initial images with hash values ​​are stored in the fast similarity vector retrieval library, and an index (vector index) is constructed by the index type provided by the FAISS tool. At this time, the hash value can be regarded as a vector. Through the above-mentioned index, according to the hash value of each initial image, the similarity value of each initial image in the fast similarity vector retrieval library is calculated, and through the density-based clustering algorithm DBSCAN (Density-Based Spatial Clustering of Applications with Noise), according to the similarity value of each initial image, all initial images in the fast similarity vector retrieval library are clustered according to the similarity value of each initial image to obtain a plurality of clusters of image sets, wherein the similarity value of the image set of each cluster is within a preset threshold, and the image sets of all clusters are aggregated to obtain an intermediate image set.

[0096] In this embodiment, by calculating hash values ​​and constructing a fast similarity vector retrieval library, the retrieval speed of image similarity can be improved, thereby improving the efficiency of model training; clustering processing is performed through a clustering algorithm, which can identify image features with similar characteristics, improve the accuracy of clustering, and also provide a basis for subsequent model training.

[0097] In another embodiment, performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture includes:

[0098] Performing discrete cosine transformation on each initial picture in the initial picture set to obtain an image transformation result;

[0099] Extracting low-frequency coefficients from the image transformation result, and calculating the mean of the low-frequency coefficients to generate a binary image;

[0100] The binary images are formatted to obtain a hash value corresponding to each of the initial images.

[0101] In this embodiment, before performing hash calculation on each initial picture in the initial picture set, it is also necessary to perform image preprocessing on the initial picture set. The image preprocessing includes normalization and grayscale. Each initial picture in the initial picture set is adjusted to a fixed size, and then converted into a grayscale image. Discrete cosine transform (DCT) is performed on the grayscale image to obtain an image transformation result of the grayscale image. Low-frequency coefficients are extracted from the image transformation result, for example, the low-frequency coefficients of 8*8 (or other sizes) in the upper left corner are extracted, and the mean of these 64 low-frequency coefficients is calculated. Each coefficient is compared with the mean, and those greater than the mean are set to 1, and those less than or equal to the mean are set to 0, thereby generating a 64-bit binary string (binary picture), and the binary string is converted according to a preset format (such as converted to hexadecimal) to obtain a hash value corresponding to each of the initial pictures.

[0102] In this embodiment, by calculating the hash value of the initial image and using the hash value as the basis for subsequent image clustering operations, the subsequent clustering efficiency can be improved, thereby improving the overall training efficiency of the consultation model. The hash value generated by the perceptual hash algorithm is sensitive to significant changes in the image (such as content tampering), and can prevent the image from being tampered with, thereby improving the accuracy of subsequent model training.

[0103] In one embodiment, performing text mapping on each target image in the target image set to obtain a text description of each target image includes:

[0104] Using a pre-trained feature extraction model to extract features from each target image in the target image set to obtain target image features;

[0105] Inputting the target image features into a pre-trained classification model, processing the target image features through multiple sub-models of the classification model, and obtaining recognition results of the multiple sub-models;

[0106] Generate binding text of the target image feature according to the recognition results of the plurality of sub-models and a preset fixed template;

[0107] The recognition results of the multiple sub-models and the bound text are aggregated to obtain a text description of each target image.

[0108] In this embodiment, the pre-trained feature extraction model can be a deep learning model such as a convolutional neural network, and the features of each target image in the target image set are extracted by the feature extraction model to obtain the target image features; the pre-trained classification model includes but is not limited to a support vector machine (SVM), a random forest model, a decision tree model or a deep neural network model, etc.; the classification model includes multiple sub-models for mapping different results to the target image features, such as four sub-models corresponding to departments, image types, image subcategories, and image texts, and each sub-model outputs a corresponding result; and a fixed template and the recognition results of the above-mentioned sub-models are used to generate binding text corresponding to the target image, and the recognition results of multiple sub-models are combined with the binding text to finally obtain a text description of each target image.

[0109] In this embodiment, the workload of manual annotation is reduced and the efficiency of the consultation model training is improved by automating the text mapping. The accuracy and reliability of the overall recognition can be further improved by combining multiple sub-models.

[0110] In another embodiment, after obtaining the text description of each target image, the method further includes:

[0111] Sending the target image and the text description of the target image to the review end for review;

[0112] If the review fails, the target image that fails the review and the text description corresponding to the target image will be deleted.

[0113] In this embodiment, in order to ensure that the pictures sent by the model comply with medical standards and prevent users from causing resistance and antipathy, it is necessary to have a real doctor review the target picture and the text description corresponding to the target picture to screen out non-compliant target pictures or non-compliant text descriptions of target pictures. Specifically, all target pictures and the text descriptions of the target pictures are sent to the review end for review. The review end reviews each target picture and the text of the target picture. If any of the target picture and the text description of the target picture fails the review, the target picture and the text description of the target picture will be deleted together.

[0114] In this embodiment, each of the target images and the text description of each target image are reviewed by the audit end to filter out non-compliant images, thereby improving user experience and improving model recognition capabilities.

[0115] In one embodiment, after obtaining the consultation information sent by the patient when receiving the online consultation, the method further includes:

[0116] Identify the data type of the medical inquiry information;

[0117] If the data type is identified as a non-text form, a text conversion method corresponding to the non-text form is obtained;

[0118] The medical consultation information is converted into a text form according to a text conversion method corresponding to the non-text form.

[0119] In this embodiment, after receiving the medical inquiry information input by the user, the data type of the medical inquiry information is queried. The specific query method can be through direct judgment. If the data type of the medical inquiry information is a string type, the medical inquiry information is judged to be in text form and no conversion is performed; if the data type of the medical inquiry information is identified as a non-text form, the conversion method corresponding to the non-text form is obtained, such as speech recognition technology (Speech-to-Text, API, etc.). When the medical inquiry information exists in the form of a file, the file extension is checked. If the file extension is .wav, etc., the medical inquiry information is judged to be an audio file; the medical inquiry information is converted into text information through the above-mentioned non-text text conversion method.

[0120] In this embodiment, by supporting multiple forms of input of medical inquiry information, the user experience can be improved, the flexibility of the query is increased, and it is convenient for patients who are not good at typing.

[0121] In one embodiment, the use of the target medical inquiry model to identify the medical inquiry information, obtain the image information and bound text corresponding to the medical inquiry information, and feed back the image information and bound text corresponding to the medical inquiry information to the patient, includes:

[0122] Receiving medical inquiry information through the input module of the target medical inquiry model, and extracting text features of the medical inquiry information and image features of each compliant image in the compliant image library through the feature extraction module of the target medical inquiry model;

[0123] Calculating a matching value between the text feature and each of the image features through a matching module of the target medical inquiry model, and determining a compliance image corresponding to the medical inquiry information from the compliance image library according to the matching value;

[0124] Extracting the binding text of the compliance image by using the binding text extraction module of the medical inquiry model;

[0125] The compliance image and the bound text of the compliance image are fed back to the patient through the output module of the medical inquiry model.

[0126] In this embodiment, the above step S205 illustrates that the medical inquiry model includes an input module, a feature extraction module, a matching module, a bound text extraction module and an output module; the preprocessed medical inquiry information is input into the target medical inquiry model through the input module, the text features of the medical inquiry information and the image features of each compliant image in the compliant image library are extracted respectively through the feature extraction module, and then the similarity values ​​of the text features and each image feature in the compliant database are calculated respectively through the matching module, wherein the calculation method includes but is not limited to cosine similarity, Euclidean distance, etc., for the calculated similarity values, the similarity values ​​are sorted in a descending order, and finally the compliant image with the highest similarity value is selected as the image corresponding to the medical inquiry information; when the compliant image is determined, the bound text corresponding to the compliant image is extracted through the bound text extraction module, and the bound text is pre-set; finally, the compliant image and the bound text corresponding to the compliant image are fed back to the patient through the output layer.

[0127] In this embodiment, the medical inquiry information is automatically processed through the target medical inquiry model, and the compliant images corresponding to the medical inquiry information and the text corresponding to the compliant images are fed back to the patient, reducing the time for manual search, and guiding the patient to express his or her condition in more precise language through the compliant images and bound texts. This not only improves the efficiency of online medical consultations, but also helps the corresponding doctors to make more accurate suggestions and treatment plans based on the condition, thereby improving the accuracy of online medical consultations.

[0128] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned online consultation results that can be published in pictures, etc., the above-mentioned online consultation results that can be published in pictures, etc. can also be stored in a blockchain node.

[0129] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0130] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0131] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0133] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0134] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of an online consultation device capable of sending pictures. Figure 2 Corresponding to the method embodiment shown, the apparatus can be specifically applied to various computer devices.

[0135] like Figure 3 As shown, the online medical consultation device 300 capable of sending images in this embodiment includes: an acquisition module 301, a clustering module 302, a screening module 303, a mapping module 304, a training module 305, and a feedback module 306. Among them:

[0136] The acquisition module 301 is used to obtain a set of consultation logs of real doctors from a preset consultation database, and filter out pictures sent by doctors to patients from the consultation logs to obtain an initial set of pictures;

[0137] A clustering module 302 is used to perform clustering processing on the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes multiple clusters;

[0138] In one embodiment, the clustering module 302 includes:

[0139] A hash calculation submodule, used to perform hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture;

[0140] A similarity value calculation submodule is used to pre-build a fast similarity vector retrieval library and an index, store the initial image with a hash value in the fast similarity vector retrieval library, traverse the fast similarity vector retrieval library, and calculate the similarity value of each initial image in the fast similarity vector retrieval library through the index;

[0141] The clustering submodule is used to cluster the fast similarity vector retrieval library according to the similarity value of each of the initial pictures using a preset clustering algorithm to obtain an intermediate picture set including a plurality of clusters.

[0142] In another embodiment, the hash calculation submodule includes:

[0143] A transformation subunit, configured to perform discrete cosine transformation on each initial picture in the initial picture set to obtain an image transformation result;

[0144] A mean calculation subunit, used for extracting low-frequency coefficients from the image transformation result, and calculating the mean of the low-frequency coefficients to generate a binary image;

[0145] The format conversion subunit is used to convert the format of the binary image to obtain a hash value corresponding to each of the initial images.

[0146] A screening module 303 is used to select a preset number of pictures from each cluster of the intermediate picture set according to the frequency of picture usage to obtain a target picture set;

[0147] A mapping module 304 is used to perform text mapping on each target image in the target image set to obtain a text description of each target image;

[0148] In one embodiment, the mapping module 304 includes:

[0149] A feature extraction submodule is used to extract features from each target image in the target image set using a pre-trained feature extraction model to obtain target image features;

[0150] A recognition submodule, used for inputting the target image features into a pre-trained classification model, processing the target image features through multiple sub-models of the classification model, and obtaining recognition results of the multiple sub-models;

[0151] A generating submodule, used for generating binding text of the target image feature according to the recognition results of the plurality of sub-models and a preset fixed template;

[0152] The summary submodule is used to summarize the recognition results of multiple sub-models and the bound text to obtain a text description of each target image.

[0153] In one embodiment, the apparatus further comprises:

[0154] A review module, used for sending the target image and the text description of the target image to the review end for review;

[0155] The deletion module is used to delete the target image and the text description corresponding to the target image that fail the review if the review fails.

[0156] A training module 305 is used to collect all target images with text descriptions to obtain training data, and use the training data to train the initial diagnosis model to obtain a target diagnosis model;

[0157] The feedback module 306 is used to obtain the consultation information sent by the patient when receiving an online consultation, identify the consultation information using the target consultation model, obtain the image information and bound text corresponding to the consultation information, and feed back the image information and bound text corresponding to the consultation information to the patient.

[0158] In one embodiment, the apparatus further comprises:

[0159] A type identification module, used to identify the data type of the medical inquiry information;

[0160] A method acquisition module, for acquiring a text conversion method corresponding to the non-text form if the data type is identified as a non-text form;

[0161] The conversion module is used to convert the medical consultation information into a text form according to a text conversion method corresponding to the non-text form.

[0162] In one embodiment, the feedback module 306 includes:

[0163] A second feature extraction submodule, configured to receive the medical inquiry information through the input module of the target medical inquiry model, and extract text features of the medical inquiry information and image features of each compliant image in the compliant image library through the feature extraction module of the target medical inquiry model;

[0164] A matching value calculation submodule, used for calculating the matching value between the text feature and each of the image features through the matching module of the target medical inquiry model, and determining the compliance image corresponding to the medical inquiry information from the compliance image library according to the matching value;

[0165] A binding text extraction submodule, used for extracting the binding text of the compliance image through the binding text extraction module of the medical inquiry model;

[0166] The binding text output submodule is used to feed back the compliance image and the binding text of the compliance image to the patient through the output module of the medical inquiry model.

[0167] In this embodiment, the pictures sent by the doctors to the patients are screened based on the medical records of the real doctors and the patients to obtain the initial picture set. Since only the pictures sent by the real doctors to the patients are screened, the compliance of the initial picture set can be guaranteed, thereby improving the accuracy of subsequent online consultations and avoiding the inclusion of low-quality pictures into the initial picture set, which would cause inaccurate subsequent model training or require additional picture compliance verification steps.

[0168] By clustering the initial image set using a preset clustering method, similar images can be grouped together, reducing data redundancy and providing more accurate training data for subsequent consultation model training, thereby improving the accuracy of subsequent consultation models.

[0169] According to the frequency of use of the pictures, a preset number of pictures are selected in each cluster to ensure that the pictures in the target picture set are highly representative and practical, which can improve the accuracy and generalization ability of the model, thereby improving the accuracy of online consultation; in the subsequent consultation model training steps, the consumption of computing resources and the occupation of storage space are reduced, thereby improving the efficiency of model training;

[0170] By performing text mapping on each target image in the target image set and obtaining a text description of each target image, the model can more accurately understand the image content and context, thereby improving training efficiency and accuracy; by not showing the user more detailed information than necessary, information overload and interference can be reduced, allowing the user to focus more on the image text and bound text, thereby improving the accuracy and efficiency of user decision-making;

[0171] By collecting all target images with text descriptions to obtain training data, the initial consultation model is trained using the training data to obtain a target consultation model. By using the screened target images with text descriptions as training data, the accuracy and compliance of the subsequent target consultation model in processing data can be improved, the accuracy and efficiency of the online consultation model can be improved, and the user experience can be improved.

[0172] The target medical inquiry model is used to identify the medical inquiry information, and the image information and bound text corresponding to the medical inquiry information are obtained, and the image information and the bound text corresponding to the medical inquiry information are fed back to the patient, which can automatically guide the patient to indicate the exact pain site. The specific pain site of the patient can be known without the guidance of the doctor, thereby improving the efficiency and accuracy of online medical consultations; guidance is provided through clear and concise pictures and bound texts, which prevents doctors from misdiagnosing due to unclear patient expressions, thereby improving the accuracy of online medical consultations, and this method can reduce the workload of doctors, thereby being able to efficiently handle more online medical consultation requests.

[0173] In order to solve the above technical problems, the present application embodiment also provides a device (computer device). Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0174] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0175] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0176] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the online consultation method that can send pictures, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0177] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the online consultation method with image publishing capability.

[0178] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0179] During the implementation of the electronic device of the present application, the pictures sent by the doctor to the patient are screened based on the medical logs of the real doctor and the patient to obtain the initial picture set. Since only the pictures sent by the real doctor to the patient are screened, the compliance of the initial picture set can be guaranteed, thereby improving the accuracy of subsequent online consultations and avoiding the inclusion of low-quality pictures into the initial picture set, which would cause inaccurate subsequent model training or require additional picture compliance verification steps;

[0180] By clustering the initial image set using a preset clustering method, similar images can be grouped together, reducing data redundancy and providing more accurate training data for subsequent consultation model training, thereby improving the accuracy of subsequent consultation models.

[0181] According to the frequency of use of the pictures, a preset number of pictures are selected in each cluster to ensure that the pictures in the target picture set are highly representative and practical, which can improve the accuracy and generalization ability of the model, thereby improving the accuracy of online consultation; in the subsequent consultation model training steps, the consumption of computing resources and the occupation of storage space are reduced, thereby improving the efficiency of model training;

[0182] By performing text mapping on each target image in the target image set and obtaining a text description of each target image, the model can more accurately understand the image content and context, thereby improving training efficiency and accuracy; by not showing the user more detailed information than necessary, information overload and interference can be reduced, allowing the user to focus more on the image text and bound text, thereby improving the accuracy and efficiency of user decision-making;

[0183] By collecting all target images with text descriptions to obtain training data, the initial consultation model is trained using the training data to obtain a target consultation model. By using the screened target images with text descriptions as training data, the accuracy and compliance of the subsequent target consultation model in processing data can be improved, the accuracy and efficiency of the online consultation model can be improved, and the user experience can be improved.

[0184] The target medical inquiry model is used to identify the medical inquiry information, and the image information and bound text corresponding to the medical inquiry information are obtained, and the image information and the bound text corresponding to the medical inquiry information are fed back to the patient, which can automatically guide the patient to indicate the exact pain site. The specific pain site of the patient can be known without the guidance of the doctor, thereby improving the efficiency and accuracy of online medical consultations; guidance is provided through clear and concise pictures and bound texts, which prevents doctors from misdiagnosing due to unclear patient expressions, thereby improving the accuracy of online medical consultations, and this method can reduce the workload of doctors, thereby being able to efficiently handle more online medical consultation requests.

[0185] The present application also provides another implementation, namely, providing a storage medium (computer-readable storage medium), wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the online consultation method that can send images as described above.

[0186] During the implementation of the computer-readable storage medium of the present application, pictures sent by doctors to patients are screened based on the medical logs of real doctors and patients to obtain an initial picture set. Since only pictures sent by real doctors to patients are screened, the compliance of the initial picture set can be guaranteed, thereby improving the accuracy of subsequent online consultations and avoiding the inclusion of low-quality pictures in the initial picture set, which would cause inaccurate subsequent model training or require additional picture compliance verification steps;

[0187] By clustering the initial image set using a preset clustering method, similar images can be grouped together, reducing data redundancy and providing more accurate training data for subsequent consultation model training, thereby improving the accuracy of subsequent consultation models.

[0188] According to the frequency of use of the pictures, a preset number of pictures are selected in each cluster to ensure that the pictures in the target picture set are highly representative and practical, which can improve the accuracy and generalization ability of the model, thereby improving the accuracy of online consultation; in the subsequent consultation model training steps, the consumption of computing resources and the occupation of storage space are reduced, thereby improving the efficiency of model training;

[0189] By performing text mapping on each target image in the target image set and obtaining a text description of each target image, the model can more accurately understand the image content and context, thereby improving training efficiency and accuracy; by not showing the user more detailed information than necessary, information overload and interference can be reduced, allowing the user to focus more on the image text and bound text, thereby improving the accuracy and efficiency of user decision-making;

[0190] By collecting all target images with text descriptions to obtain training data, the initial consultation model is trained using the training data to obtain a target consultation model. By using the screened target images with text descriptions as training data, the accuracy and compliance of the subsequent target consultation model in processing data can be improved, the accuracy and efficiency of the online consultation model can be improved, and the user experience can be improved.

[0191] The target medical inquiry model is used to identify the medical inquiry information, and the image information and bound text corresponding to the medical inquiry information are obtained, and the image information and the bound text corresponding to the medical inquiry information are fed back to the patient, which can automatically guide the patient to indicate the exact pain site. The specific pain site of the patient can be known without the guidance of the doctor, thereby improving the efficiency and accuracy of online medical consultations; guidance is provided through clear and concise pictures and bound texts, which prevents doctors from misdiagnosing due to unclear patient expressions, thereby improving the accuracy of online medical consultations, and this method can reduce the workload of doctors, thereby being able to efficiently handle more online medical consultation requests.

[0192] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

[0193] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0194] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An online consultation method capable of sending pictures, characterized in that: The steps include: Obtaining a set of consultation logs of real doctors from a preset consultation database, and filtering out pictures sent by doctors to patients from the consultation logs to obtain an initial set of pictures; Performing clustering processing on the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes a plurality of clusters; Based on each cluster of the intermediate picture set, a preset number of pictures are selected from each cluster according to the frequency of use of the pictures to obtain a target picture set; Performing text mapping on each target image in the target image set to obtain a text description of each target image; Collect all target images with text descriptions to obtain training data, and use the training data to train the initial consultation model to obtain the target consultation model; When an online consultation is received, the consultation information sent by the patient is obtained, the consultation information is identified using the target consultation model, the image information and bound text corresponding to the consultation information are obtained, and the image information and bound text corresponding to the consultation information are fed back to the patient.

2. The online medical consultation method capable of sending pictures as claimed in claim 1, characterized in that: The clustering process is performed on the initial picture set to obtain an intermediate picture set, including: Performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture; Pre-building a fast similarity vector retrieval library and an index, storing the initial image with a hash value in the fast similarity vector retrieval library, traversing the fast similarity vector retrieval library, and calculating the similarity value of each initial image in the fast similarity vector retrieval library through the index; The fast similarity vector retrieval library is clustered according to the similarity value of each of the initial pictures using a preset clustering algorithm to obtain an intermediate picture set including a plurality of clusters.

3. The online medical consultation method capable of sending pictures as claimed in claim 2, characterized in that: The performing hash calculation on each initial picture in the initial picture set by using a preset hash algorithm to obtain a hash value corresponding to each initial picture includes: Performing discrete cosine transformation on each initial picture in the initial picture set to obtain an image transformation result; Extracting low-frequency coefficients from the image transformation result, and calculating the mean of the low-frequency coefficients to generate a binary image; The binary images are formatted to obtain a hash value corresponding to each of the initial images.

4. The online medical consultation method capable of sending pictures as claimed in claim 1, characterized in that: The performing text mapping on each target image in the target image set to obtain a text description of each target image includes: Using a pre-trained feature extraction model to extract features from each target image in the target image set to obtain target image features; Inputting the target image features into a pre-trained classification model, processing the target image features through multiple sub-models of the classification model, and obtaining recognition results of the multiple sub-models; Generate binding text of the target image feature according to the recognition results of the plurality of sub-models and a preset fixed template; The recognition results of the multiple sub-models and the bound text are aggregated to obtain a text description of each target image.

5. The online medical consultation method capable of sending pictures according to claim 1 or 4, characterized in that: After obtaining the text description of each target image, the method further includes: Sending the target image and the text description of the target image to the review end for review; If the review fails, the target image that fails the review and the text description corresponding to the target image will be deleted.

6. The online medical consultation method capable of sending pictures as claimed in claim 1, characterized in that: After obtaining the consultation information sent by the patient when receiving the online consultation, the method further includes: Identify the data type of the medical inquiry information; If the data type is identified as a non-text form, a text conversion method corresponding to the non-text form is obtained; and according to the text conversion method corresponding to the non-text form, the medical consultation information is converted into a text form.

7. The online medical consultation method capable of sending pictures as claimed in claim 1, characterized in that: The using the target medical inquiry model to identify the medical inquiry information, obtaining the picture information and the bound text corresponding to the medical inquiry information, and feeding back the picture information and the bound text corresponding to the medical inquiry information to the patient, includes: Receiving medical inquiry information through the input module of the target medical inquiry model, and extracting text features of the medical inquiry information and image features of each compliant image in the compliant image library through the feature extraction module of the target medical inquiry model; Calculating a matching value between the text feature and each of the image features through a matching module of the target medical inquiry model, and determining a compliance image corresponding to the medical inquiry information from the compliance image library according to the matching value; Extracting the binding text of the compliance image by using the binding text extraction module of the medical inquiry model; The compliance image and the bound text of the compliance image are fed back to the patient through the output module of the medical inquiry model.

8. An online medical consultation device capable of sending pictures, characterized in that: The device comprises: An acquisition module is used to acquire a set of consultation logs of real doctors from a preset consultation database, and to filter out pictures sent by doctors to patients from the consultation logs to obtain an initial set of pictures; A clustering module, configured to perform clustering processing on the initial picture set to obtain an intermediate picture set, wherein the intermediate picture set includes a plurality of clusters; A screening module, configured to select a preset number of pictures from each cluster of the intermediate picture set according to the frequency of picture usage, to obtain a target picture set; A mapping module, used to perform text mapping on each target image in the target image set to obtain a text description of each target image; A training module is used to collect all target images with text descriptions to obtain training data, and use the training data to train the initial consultation model to obtain a target consultation model; The feedback module is used to obtain the inquiry information sent by the patient when receiving an online consultation, identify the inquiry information using the target inquiry model, obtain the picture information and bound text corresponding to the inquiry information, and feed back the picture information and bound text corresponding to the inquiry information to the patient.

9. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the online medical consultation method that can publish images as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the online medical consultation method capable of publishing images as described in any one of claims 1 to 7 is implemented.

Images