Remote medical system and method based on 5G technology

Abstract

The invention relates to the field of intelligent analysis of medical images, and provides a remote medical system and method based on a 5G technology, which adopts a data analysis technology based on artificial intelligence to judge the fracture type of a patient by analyzing information contained in a bone X-ray diagram of the patient in a high-dimensional feature space. The method is used for assisting telemedicine experts in quickly making judgments. Therefore, a doctor can be helped to judge the fracture type more accurately, the workload of the doctor can be relieved, and the accuracy and efficiency of diagnosis can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent analysis of medical images, and more specifically, to a telemedicine system and method based on 5G technology. Background Art

[0002] At present, there are problems such as uneven regional medical development levels, lack of quality professional guidance for grassroots medical staff in marginal areas, insufficient medical resources, and shortage of senior doctors. It is difficult to make a quick diagnosis in the face of relatively complex clinical data. In addition, there are large individual differences among patients, atypical examination results, false negatives, etc. Physicians often miss or misdiagnose under high clinical workload. Especially in the diagnosis of bone diseases, conventional bone interpretation is relatively easy, but there are still deficiencies in the diagnosis of some irregular bones. Some fractures have always been difficult to diagnose and prone to missed diagnosis in the field of orthopedics due to their complex anatomy and hidden fracture lines.

[0003] Therefore, an optimized telemedicine solution based on 5G technology is needed. Summary of the invention

[0004] In view of the shortcomings of the prior art, this application provides a 5G-based telemedicine system and method, which uses artificial intelligence-based data analysis technology to analyze the information contained in the patient's bone X-ray image in a high-dimensional feature space to determine the patient's fracture type, thereby assisting telemedicine experts in making a quick judgment. In this way, it can help doctors to more accurately determine the type of fracture, which can not only reduce the doctor's workload, but also improve the accuracy and efficiency of diagnosis.

[0005] According to one aspect of the present application, a telemedicine system based on 5G technology is provided, which includes:

[0006] A bone-related data acquisition module is used to obtain a bone X-ray image of the patient;

[0007] A bone-related data preprocessing module, used for performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image;

[0008] A bone-related data feature encoding module, used for performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector;

[0009] The fracture type result generation module is used to determine the patient's fracture type based on the bone condition detection feature vector.

[0010] According to another aspect of the present application, a telemedicine method based on 5G technology is provided, which includes:

[0011] Obtain a skeletal X-ray of the patient;

[0012] Performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image;

[0013] Performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector;

[0014] Based on the bone condition detection feature vector, the patient's fracture type is determined.

[0015] This application has significant technical effects due to the adoption of the above technical solutions:

[0016] The 5G-based telemedicine system and method provided in this application uses artificial intelligence-based data analysis technology to analyze the information contained in the patient's bone X-ray image in a high-dimensional feature space to determine the patient's fracture type, thereby assisting telemedicine experts in making a quick judgment. In this way, it can help doctors to more accurately determine the type of fracture, which can not only reduce the doctor's workload, but also improve the accuracy and efficiency of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a flowchart of a telemedicine method based on 5G technology according to an embodiment of the present application.

[0019] Figure 2 This is a flowchart of performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector in a telemedicine method based on 5G technology according to an embodiment of the present application.

[0020] Figure 3 This is a flowchart of performing detailed feature extraction on the X-ray image of the bone region of interest to obtain the bone condition detection feature vector in the telemedicine method based on 5G technology according to an embodiment of the present application.

[0021] Figure 4 This is a system block diagram of a telemedicine system based on 5G technology according to an embodiment of the present application.

[0022] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] In today's society, the uneven distribution of medical resources has become an issue that cannot be ignored. Especially in some remote areas, primary medical institutions often face difficulties such as shortage of professionals, outdated medical equipment, and insufficient technical guidance. These problems not only limit the quality of local medical services, but also seriously affect the health and well-being of patients.

[0025] Faced with the growing medical needs, grassroots medical staff often work hard to cope with various clinical challenges under high workload. However, due to the lack of sufficient professional training and advanced medical equipment, they are often unable to cope with complex cases. Especially in the field of bone disease diagnosis, although conventional bone imaging interpretation is relatively easy, when faced with some cases with complex anatomical structures and unclear fracture lines, the difficulty of diagnosis and the risk of missed diagnosis and misdiagnosis are greatly increased.

[0026] In addition, individual differences among patients, atypical examination results, and false negatives also bring additional challenges to clinical diagnosis. In this context, physicians are inevitably prone to omissions under heavy workload, which undoubtedly poses a potential threat to the treatment and rehabilitation of patients. Therefore, a telemedicine solution based on 5G technology is expected.

[0027] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks provides new solutions and solutions for telemedicine based on 5G technology.

[0028] Figure 1 Flow chart of a telemedicine method based on 5G technology according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the telemedicine method based on 5G technology includes: S110, obtaining a bone X-ray image of the patient; S120, performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image; S130, performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector; S140, judging the patient's fracture type based on the bone condition detection feature vector.

[0029] Specifically, in the technical solution of the present application, it is first necessary to obtain the patient's bone X-ray image. It should be understood that X-rays can penetrate soft tissues but not bones, so the bone X-ray image can clearly show the outline, morphology and structure of the bones, and this information is the basis for judging the health status of the bones. When a bone fracture occurs, the fracture line will appear as an interruption or dislocation of bone continuity in the X-ray image, and different types of fractures (such as transverse fractures, oblique fractures, spiral fractures, comminuted fractures, etc.) will show different fracture line morphology and fracture site characteristics in the X-ray image. After the fracture, the fracture ends may be displaced or angulated, and these changes will also be clearly displayed in the X-ray image. In general, the bone X-ray image is the most direct and reliable evidence for judging the type of fracture. By analyzing the fracture line morphology, fracture site, displacement and angulation in the X-ray image, the type and severity of the fracture can be determined.

[0030] In an embodiment of the present application, a specific implementation of a patient's bone X-ray image can be: obtaining a bone X-ray image from the imaging system of a relevant medical institution, and the medical institution will use the high speed and low latency characteristics of 5G technology to transmit these images to the telemedicine platform in real time or quasi-real time. The application of 5G technology can ensure the stability and clarity of image transmission, thereby providing high-quality image data for telemedicine experts. It is worth noting that during the data transmission process, medical institutions and telemedicine platforms need to strictly abide by relevant data security and privacy protection regulations to ensure that patients' personal information and medical data are not leaked or abused.

[0031] Next, the bone X-ray image is subjected to image preprocessing to obtain a preprocessed bone X-ray image. It should be understood that the noise in the bone X-ray image comes from a variety of factors, including electronic noise of the equipment, inhomogeneity of the detector, motion artifacts in the patient's body, etc. The presence of noise will reduce the clarity of the image, making it difficult to identify details such as fracture lines and fracture sites, thereby affecting the accurate judgment of the fracture type. In addition, in the bone X-ray image, insufficient contrast will cause the contrast between the fracture line and the surrounding bone tissue to be low, which will also increase the difficulty of judging the fracture type, because subtle fracture lines or fracture sites may be difficult to detect in images with insufficient contrast. In order to solve these problems, image preprocessing is particularly important. Through preprocessing operations such as noise removal and contrast enhancement, the quality of the image can be improved, and key features such as fracture lines and fracture sites can be more prominent, thereby improving the accuracy of fracture type judgment.

[0032] In an embodiment of the present application, a possible implementation method for performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image may be: a. De-noising of the bone X-ray image: applying the cv2.GaussianBlur() function in OpenCV to implement Gaussian filtering denoising; b. Histogram equalization of the bone X-ray image: applying the cv2.equalizeHist() function to implement.

[0033] Figure 2 This is a flowchart of performing content feature encoding on the pre-processed bone X-ray image to obtain a bone condition detection feature vector in a telemedicine method based on 5G technology according to an embodiment of the present application. Figure 2 As shown, the content feature encoding of the preprocessed bone X-ray image to obtain a bone condition detection feature vector includes: S131, performing bone target detection on the preprocessed bone X-ray image to obtain a bone region of interest X-ray image; S132, performing detail feature extraction on the bone region of interest X-ray image to obtain the bone condition detection feature vector.

[0034] Next, the pre-processed bone X-ray image is passed through the bone region target detection network to obtain a bone region of interest X-ray image. The bone region target detection network can automatically identify and extract the bone region in the X-ray image, eliminate the interference of the non-bone part, and enable the subsequent processing algorithm to be more focused on the diagnosis and analysis of the bone part. Moreover, before the subsequent fracture type judgment is further analyzed, the bone region is first determined by the target detection network, which can greatly reduce the amount of data to be processed, thereby improving the overall processing efficiency. It should be understood that the bone region target detection network in this application is a target detection network based on an anchor window. Anchor window is a concept commonly used in target detection algorithms, which pre-sets a series of rectangular frames (also called anchor points or prior frames) of fixed size and aspect ratio, which are used to predict the position and shape of the target in the image. Anchor window can help the algorithm locate the approximate position of the target faster, reduce the search space, and improve detection efficiency. At the same time, by adjusting the size and shape of the anchor window, it can adapt to target detection tasks of different sizes and shapes.

[0035] In an embodiment of the present application, a possible implementation method of passing the preprocessed skeleton X-ray image through a skeleton region target detection network to obtain a skeleton region of interest X-ray image may be: using the skeleton region target detection network to process the preprocessed skeleton X-ray image using the following formula to obtain the skeleton region of interest X-ray image; wherein the formula is:

[0036] ROI=(cls(ψ det ,B),Regr(ψ det,B))

[0037] Among them, ψ det is the preprocessed bone X-ray image, B is the anchor frame, ROI is the bone region of interest X-ray image, cls(ψ det ,B) indicates classification, Regr(ψ det ,B) indicates regression.

[0038] Figure 3 This is a flowchart of extracting detailed features from the X-ray image of the bone region of interest to obtain the bone condition detection feature vector in the telemedicine method based on 5G technology according to an embodiment of the present application. Figure 3 As shown, the detailed feature extraction of the X-ray image of the bone region of interest to obtain the bone condition detection feature vector includes: S1321, passing the X-ray image of the bone region of interest through a bone feature encoder to obtain a bone region feature map; S1322, pooling the bone region feature map along the channel dimension to obtain a bone region feature vector; S1323, passing the bone region feature map through a bone space feature enhancer to obtain a bone key region enhanced feature map; S1324, pooling the bone key region enhanced feature map along the channel dimension to obtain a bone key region enhanced feature vector; S1325, based on the bone region feature vector, performing reverse optimization adjustment on the bone key region enhanced feature vector based on a multi-angle feature association metric to obtain an optimized bone key region enhanced feature vector; S1326, fusing the bone region feature vector and the optimized bone key region enhanced feature vector to obtain the bone condition detection feature vector.

[0039] The original X-ray image contains a large amount of pixel information, but not all information is useful for judging the fracture type. In order to filter out the most representative and discriminative information for judging the fracture type from these data, i.e., "features", it is necessary to pass the bone region X-ray image through the bone feature encoder to obtain the bone region feature map. In the process of processing by the bone feature encoder, attention should be paid to the morphological characteristics of the bone, such as the outline, texture, density change, position, shape and direction of the fracture line of the bone, etc. These features are crucial for judging the fracture type. In this application, the bone feature encoder is a convolutional neural network model including a deep and shallow feature fusion module. The shallow features extracted from the bone region X-ray image by the convolutional neural network model (CNN) mainly include low-level information such as edges and textures, which are very important for describing the basic morphology and surface features of the bone; the deep features extracted from the bone region X-ray image mainly include the shape, structure and possible injury pattern of the bone, which have higher discrimination and representativeness for judging the fracture type. Through the deep and shallow feature fusion module, CNN can simultaneously take advantage of the advantages of shallow features and deep features to form a more comprehensive and accurate bone feature representation.

[0040] In an embodiment of the present application, a possible implementation method of passing the skeleton region of interest X-ray image through a skeleton feature encoder to obtain a skeleton region feature map may be: extracting a skeleton region shallow feature map from the i-th layer of the skeleton feature encoder, the i-th layer being the first to sixth layers of the first convolutional neural network model; extracting a skeleton region deep feature map from the j-th layer of the skeleton feature encoder, the ratio between the j-th layer and the i-th layer being greater than or equal to 5; and using the deep and shallow feature fusion module to fuse the skeleton region shallow feature map and the skeleton region deep feature map to obtain the skeleton region feature map.

[0041] The skeleton region feature map usually contains a large amount of information, but not all of the information is useful for judging the fracture type. In order to remove redundant information and retain important skeleton region feature information, the skeleton region feature map is pooled along each feature matrix of the channel dimension to obtain the skeleton region feature vector in the technical solution of the present application. Through the pooling operation, the scale of processing data is reduced, thereby reducing the calculation cost and improving the processing speed. And the feature vector extracted based on the pooling operation can more accurately reflect the structural characteristics of the skeleton, thereby providing a more reliable basis for the classification of the fracture type, which helps to improve the precision and accuracy of the classification.

[0042] Although the skeleton region feature map has included the basic morphology and structural information of the skeleton, in practical applications, the information of certain regions (such as fracture occurrence, joint joints, etc.) is more critical for the judgment of fracture type. In order to strengthen the features of these key areas, these areas are more prominent in subsequent classification and recognition tasks, so the skeleton region feature map is strengthened by the skeleton space feature intensifier to obtain the skeleton key area strengthening feature map. In the present application, the skeleton space feature intensifier is a convolutional neural network model using a spatial attention mechanism. The convolutional neural network model using the spatial attention mechanism can automatically learn and identify the key areas in the skeleton region feature map, which are often closely related to the judgment of fracture type, such as fracture lines, fracture fragments, etc. By strengthening the features of these areas, the model can be more focused on these key information, thereby improving the accuracy of fracture type judgment. Specifically, the spatial attention mechanism allows the model to dynamically allocate weights on the feature map, so that the features of the important areas get more attention, so that the characteristic performance of fracture can be captured more accurately.

[0043] Although the bone key area enhancement feature map highlights the key area features directly related to the fracture, its data volume is still large, so the bone key area enhancement feature map is pooled along the channel dimension to obtain the bone key area enhancement feature vector. Through the pooling operation, each feature matrix can be downsampled along the channel dimension to reduce the spatial dimension of the data, thereby extracting a more compact and representative feature vector of the bone key area.

[0044] In particular, in the technical solution of the present application, considering that the skeleton space feature enhancer may be more focused on highlighting certain specific areas or features in the skeleton, this may cause the skeleton key area enhancement feature vector to be overemphasized in some aspects, while other equally important information may be ignored or weakened. When performing global pooling, the skeleton region feature map and the skeleton key area enhancement feature map may produce different pooling results due to different feature distributions and importance. If the pooling operation does not properly balance these two types of features, it may cause the skeleton key area enhancement feature vector to have a posterior-prior causal association missing relative to the skeleton region feature vector. In order to solve this problem, in the technical solution of the present application, based on the skeleton region feature vector, the skeleton key area enhancement feature vector is reversely optimized and adjusted based on a multi-angle feature association metric to obtain an optimized skeleton key area enhancement feature vector.

[0045] Specifically, in an embodiment of the present application, based on the bone region feature vector, the bone key region enhancement feature vector is reversely optimized and adjusted based on a multi-angle feature association metric to obtain an optimized bone key region enhancement feature vector, including: calculating the positional difference between the bone key region enhancement feature vector and the bone region feature vector to obtain a first viewpoint metric representation vector; calculating the positional point multiplication between the bone key region enhancement feature vector and the bone region feature vector to obtain a second viewpoint metric representation vector; calculating the positional sum between the bone key region enhancement feature vector and the bone region feature vector to obtain a third viewpoint metric Representation vector; calculate the gradient amplitude of each position in the first viewpoint measurement representation vector, the second viewpoint measurement representation vector and the third viewpoint measurement representation vector to obtain the first viewpoint measurement perception vector, the second viewpoint measurement perception vector and the third viewpoint measurement perception vector; input the first viewpoint measurement perception vector, the second viewpoint measurement perception vector and the third viewpoint measurement perception vector into the backward machine learning reverse multi-viewpoint perception optimization module to obtain the reverse multi-viewpoint perception optimization representation vector; calculate the position point multiplication between the reverse multi-viewpoint perception optimization representation vector and the skeleton key area enhancement feature vector to obtain the optimized skeleton key area enhancement feature vector.

[0046] More specifically, in the embodiment of the present application, the backward machine learning reverse multi-view perception optimization module is expressed by the formula:

[0047]

[0048] in, and represents the eigenvalues ​​of the i-th position of the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, respectively, and μ represents the three eigenvalues ​​of the corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector , σ is the standard deviation of the three eigenvalues ​​at corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, represents the maximum value of the three eigenvalues ​​at the corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, ReLU represents a linear rectification function, softmax represents a normalized exponential function, and v ri represents the eigenvalue of the i-th position of the inverse multi-view perceptual optimization representation vector, and exp(·) represents an exponential function with the natural constant e as the base.

[0049] Specifically, in the technical solution of the present application, based on the bone region feature vector, the bone key region enhancement feature vector is reversely optimized and adjusted based on multi-angle feature association measurement, and the process includes first calculating the position difference, point multiplication and sum between the bone key region enhancement feature vector and the bone region feature vector through multi-view analysis (Multi-View Analysis) to obtain three viewpoint measurement representation vectors, and quantify the relationship between features from different angles. Then, the gradient amplitude of each position in these viewpoint measurement representation vectors is calculated through backward error perception to identify key change points to provide a basis for feature optimization.

[0050] Then, the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector are input into the backward machine learning reverse multi-viewpoint perception optimization module, wherein the backward machine learning reverse multi-viewpoint perception optimization module captures the significant comprehensive ability factor of the feature importance perception (Feature Importance Perception) of the feature sub-dimension distribution of the corresponding positions in the first to third viewpoint metric perception vectors by adopting a squeeze-incentive mechanism composed of -functions. Then, the position point multiplication between the reverse multi-viewpoint perception optimization representation vector and the skeleton key area enhancement feature vector is calculated to perform backward optimization perception (Backward Optimization Perception) on the skeleton key area enhancement feature vector to adjust the skeleton key area enhancement feature vector so as to better associate it with the skeleton area feature vector, thereby enhancing the causal relationship between the skeleton key area enhancement feature vector and the skeleton area feature vector.

[0051] The skeleton region feature vector provides the overall morphology and structural information of the skeleton, which is the basis for judging the fracture type. The optimization of the skeleton key area strengthening feature vector further highlights the key areas directly related to the fracture through the spatial attention mechanism, and enhances the feature representation of these areas. In order to make full use of the information of both and form a more complete and in-depth understanding of the skeleton situation, the skeleton region feature vector and the optimization of the skeleton key area strengthening feature vector are fused in the technical scheme of the present application to obtain the skeleton situation detection feature vector. It should be understood that the fusion of these two feature vectors can realize the complementarity of information, so that the final skeleton situation detection feature vector not only includes the overall morphology and structural information of the skeleton, but also highlights the key areas directly related to the fracture.

[0052] In an embodiment of the present application, a possible implementation method of fusing the bone region feature vector and the optimized bone key region enhancement feature vector to obtain the bone condition detection feature vector may be: fusing the bone region feature vector and the optimized bone key region enhancement feature vector using the following fusion formula to obtain the bone condition detection feature vector; wherein the fusion formula is:

[0053]

[0054] Wherein, V1 is the bone condition detection feature vector, V a is the bone region feature vector, V b To optimize the bone key area strengthening feature vector, It represents the addition of the elements at corresponding positions of the bone area feature vector and the optimized bone key area enhancement feature vector, and α and β are weighted parameters for controlling the balance between the bone area feature vector and the optimized bone key area enhancement feature vector in the bone condition detection feature vector.

[0055] Finally, the bone condition detection feature vector is passed through a fracture type label classifier to obtain a classification result, and the classification result is used to represent the patient's fracture type. Although the bone condition detection feature vector already contains rich bone structure and morphological information, this information does not correspond to a specific fracture type label. Therefore, it is necessary to pass the bone condition detection feature vector through a fracture type label classifier. The fracture type label classifier is a machine learning model that is trained to identify and classify features in the bone condition detection feature vector to output the corresponding fracture type label. During the training process, the classifier learns how to associate specific patterns in the feature vector with known fracture type labels. Once the training is completed, the classifier can predict the new bone condition detection feature vector and output the most likely fracture type label. Through the automated classification process, doctors can obtain preliminary judgment results of the patient's fracture type more quickly. That is, the classification results provide doctors with important reference information to help them diagnose fractures more accurately, thereby saving time and improving diagnostic efficiency.

[0056] In an embodiment of the present application, the bone condition detection feature vector is passed through a fracture type label classifier to obtain a classification result, and one possible implementation method for the classification result to represent the patient's fracture type may be: the bone condition detection feature vector is processed using the fracture type label classifier with the following classification formula to generate the classification result; wherein the classification formula is:

[0057] O=softmax{(M c ,B c )|V1}

[0058] Wherein O is the classification result, V1 represents the bone condition detection feature vector, M c is the weight matrix of the fully connected layer, B c Represents the bias vector of the fully connected layer, and softmax is the normalized exponential function.

[0059] In summary, the 5G-based telemedicine method according to the embodiment of the present application is explained, which uses artificial intelligence-based data analysis technology to analyze the information contained in the patient's bone X-ray image in a high-dimensional feature space to determine the patient's fracture type, so as to assist telemedicine experts in making a quick judgment. In this way, it can help doctors to more accurately determine the type of fracture, which can not only reduce the doctor's workload, but also improve the accuracy and efficiency of diagnosis.

[0060] Figure 4 : is a system block diagram of a telemedicine system based on 5G technology according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the telemedicine system 100 based on 5G technology includes: a bone-related data acquisition module 110, used to obtain a bone X-ray image of a patient; a bone-related data preprocessing module 120, used to perform image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image; a bone-related data feature encoding module 130, used to perform content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector; a fracture type result generation module 140, used to determine the patient's fracture type based on the bone condition detection feature vector.

[0061] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the above-mentioned 5G-based telemedicine system 100 have been described in detail above. Figures 1 to 3 The description of the 5G technology-based telemedicine method has been introduced in detail, and therefore, its repeated description will be omitted.

[0062] In summary, the 5G-based telemedicine system 100 according to the embodiment of the present application is described, which uses artificial intelligence-based data analysis technology to analyze the information contained in the patient's bone X-ray image in a high-dimensional feature space to determine the patient's fracture type, so as to assist telemedicine experts in making a quick judgment. In this way, it can help doctors to more accurately determine the type of fracture, which can not only reduce the doctor's workload, but also improve the accuracy and efficiency of diagnosis.

[0063] As described above, the telemedicine system 100 based on 5G technology according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for telemedicine based on 5G technology. In one example, the telemedicine system 100 based on 5G technology according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the telemedicine system 100 based on 5G technology can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the telemedicine system 100 based on 5G technology can also be one of the many hardware modules of the wireless terminal.

[0064] Alternatively, in another example, the telemedicine system 100 based on 5G technology and the wireless terminal may also be separate devices, and the telemedicine system 100 based on 5G technology may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0065] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application. Figure 5 1 is a block diagram of an electronic device according to an embodiment of the present application. For example, the electronic device 10 can be provided as a server or a terminal device. Figure 5 The electronic device 10 includes a processing component 11, which further includes one or more processors, and a memory resource represented by a memory 12 for storing instructions executable by the processing component 11, such as an application. The application stored in the memory 12 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 11 is configured to execute instructions to perform the above method.

[0066] The electronic device 10 may further include a power supply component 13 configured to perform power management of the electronic device 10, a wired or wireless network interface 14 configured to connect the electronic device 10 to a network, and an input / output (I / O) interface 15. The electronic device 10 may operate based on an operating system stored in the memory 12, such as Windows Server™, Mac OSX™, Unix™, Linux™, FreeBSD™ or the like.

[0067] In an embodiment of the present application, a computer-readable storage medium is also provided, such as a memory 12 including computer program instructions. The computer program instructions can be executed by a processing component 11 of an electronic device 10 to complete the above method.

[0068] The present application may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer program instructions for causing a processor to implement various aspects of the present application.

[0069] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0070] The computer program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer program instructions from the network and forwards the computer program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0071] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer program instructions can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit can execute the computer program instructions, thereby realizing various aspects of the present application.

[0072] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions.

[0073] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0074] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

Claims

1. A telemedicine method based on 5G technology, characterized in that: include: Obtain a skeletal X-ray of the patient; Performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image; Performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector; Based on the bone condition detection feature vector, the patient's fracture type is determined.

2. The telemedicine method based on 5G technology according to claim 1, characterized in that: The preprocessed bone X-ray image is encoded with content features to obtain a bone condition detection feature vector, including: Performing bone target detection on the preprocessed bone X-ray image to obtain a bone region of interest X-ray image; Detail features are extracted from the X-ray image of the bone region of interest to obtain the bone condition detection feature vector.

3. The 5G-based telemedicine method according to claim 2, characterized in that: Performing bone target detection on the preprocessed bone X-ray image to obtain a bone region of interest X-ray image, including: passing the preprocessed bone X-ray image through a bone region target detection network to obtain the bone region of interest X-ray image.

4. The 5G-based telemedicine method according to claim 3, characterized in that: Extracting detailed features from the X-ray image of the bone region of interest to obtain the bone condition detection feature vector includes: Passing the bone region of interest X-ray image through a bone feature encoder to obtain a bone region feature image; Pooling the skeleton region feature map with each feature matrix along the channel dimension to obtain a skeleton region feature vector; The bone region feature map is passed through a bone space feature enhancer to obtain a bone key region enhanced feature map; Pooling the feature matrices of the bone key area enhancement feature map along the channel dimension to obtain a bone key area enhancement feature vector; Based on the bone region feature vector, the bone key region enhancement feature vector is reversely optimized and adjusted based on multi-angle feature association measurement to obtain an optimized bone key region enhancement feature vector; The bone region feature vector and the optimized bone key region enhancement feature vector are fused to obtain the bone condition detection feature vector.

5. The 5G-based telemedicine method according to claim 4, characterized in that: The skeleton region target detection network is an anchor window-based target detection network.

6. The 5G-based telemedicine method according to claim 5, characterized in that: The skeleton feature encoder is a convolutional neural network model including a deep and shallow feature fusion module, and the skeleton spatial feature enhancer is a convolutional neural network model using a spatial attention mechanism.

7. The 5G-based telemedicine method according to claim 6, characterized in that: Based on the bone region feature vector, the bone key region enhancement feature vector is reversely optimized and adjusted based on multi-angle feature association measurement to obtain an optimized bone key region enhancement feature vector, including: Calculating the position difference between the skeleton key area enhancement feature vector and the skeleton area feature vector to obtain a first viewpoint metric representation vector; Calculate the position point multiplication between the skeleton key area enhancement feature vector and the skeleton area feature vector to obtain a second viewpoint metric representation vector; Calculate the positional sum of the skeleton key area enhancement feature vector and the skeleton area feature vector to obtain a third viewpoint metric representation vector; Calculating the gradient magnitude of each position in the first viewpoint metric representation vector, the second viewpoint metric representation vector, and the third viewpoint metric representation vector to obtain a first viewpoint metric perception vector, a second viewpoint metric perception vector, and a third viewpoint metric perception vector; Inputting the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector into a backward machine learning inverse multi-view perception optimization module to obtain a reverse multi-view perception optimization representation vector; The optimized skeleton key area enhancement feature vector is obtained by calculating the position point multiplication between the reverse multi-view perception optimization representation vector and the skeleton key area enhancement feature vector.

8. The 5G-based telemedicine method according to claim 7, characterized in that: Based on the bone condition detection feature vector, the fracture type of the patient is determined, including: passing the bone condition detection feature vector through a fracture type label classifier to obtain a classification result, and the classification result is used to represent the fracture type of the patient.

9. A telemedicine system based on 5G technology, characterized in that: include: A bone-related data acquisition module is used to obtain a bone X-ray image of the patient; A bone-related data preprocessing module, used for performing image preprocessing on the bone X-ray image to obtain a preprocessed bone X-ray image; A bone-related data feature encoding module, used for performing content feature encoding on the preprocessed bone X-ray image to obtain a bone condition detection feature vector; The fracture type result generation module is used to determine the patient's fracture type based on the bone condition detection feature vector.

10. The 5G-based telemedicine system according to claim 9, characterized in that: The bone-related data preprocessing module is used to: pass the preprocessed bone X-ray image through a bone region target detection network to obtain the bone region of interest X-ray image.

Images