Multidisciplinary consultation method, device and system based on real-time point cloud rendering
Real-time point cloud rendering is performed through cloud-connected augmented reality devices, which solves the problems of communication barriers and insufficient information exchange in multidisciplinary consultations, realizes real and comprehensive information exchange between multiple experts, and improves the effectiveness of diagnosis and treatment .
Patent Information
- Application Number
- CN202411038144.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In multidisciplinary consultations, communication barriers may exist because experts come from different departments and fields, and limitations on the time and location of the consultation, making it difficult to ensure that all participants can discuss and exchange opinions in full and in-depth manner.
Through a multidisciplinary consultation method based on real-time point cloud rendering, multiple augmented reality devices connected to the cloud can obtain and transmit three-dimensional point cloud data for real-time rendering and display, allowing multiple experts to interact and interact in the virtual space with voice and virtual interaction in the virtual space , ensure that all participants have a true and comprehensive understanding of patient information.
It has achieved full and in-depth communication between multiple experts, improved the effectiveness of diagnosis and treatment, and ensured that all participants can truly and comprehensively understand the patient information, thereby assisting in obtaining comprehensive diagnostic results.
Smart Images

Figure CN118969247A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical technology, and in particular to a multidisciplinary consultation method, device and system based on real-time point cloud rendering. Background Art
[0002] Multidisciplinary consultation (MDT) is a medical model that brings together the expertise and skills of multiple disciplines to jointly diagnose and treat complex diseases. It involves experts from multiple disciplines working together as a team, usually three departments or more, with a patient-centered approach to evaluate, diagnose and treat the patient's complex conditions, with the goal of improving treatment outcomes and optimizing medical resources. In the process of multidisciplinary consultation, it is necessary to collect the patient's medical history, physical examination results, laboratory tests and imaging data. In different independent spaces, the consultation team members use a 3D scanner to obtain a point cloud data set of the human body surface, which contains not only location information but also color (RGB) information. These data sets are transmitted to each member's independent space, rendered in real time in the augmented reality device, and the patient's relevant information is also obtained through real-time scanning by the scanner.
[0003] The scanning space of each independent augmented reality device contains the surface point cloud rendering 3D model of the participants and the 3D scanning model of the patient's related examination data. After wearing the augmented reality device, the doctor can grab and rotate the data model in the space and the 3D organ model reconstructed from the patient's imaging data at will, realizing 360-degree observation and real-time communication and discussion.
[0004] Since the experts and team members involved in multidisciplinary consultations come from different departments and fields, they may have different professional backgrounds and terminology systems, which may lead to communication barriers. In addition, due to the limitations of consultation time and location, it is sometimes difficult to ensure that all participants can fully and deeply discuss and exchange opinions. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide a multidisciplinary consultation method, device and system based on real-time point cloud rendering, so that multiple experts can understand the patient's information more realistically and comprehensively, and thus conduct full and in-depth discussions and exchanges of opinions.
[0006] In a first aspect, an embodiment of the present invention provides a multidisciplinary consultation method based on real-time point cloud rendering, the method being applied to a first augmented reality device of a multidisciplinary consultation system, the multidisciplinary consultation system comprising multiple augmented reality devices, and the multiple augmented reality devices being interconnected via a cloud, the first augmented reality device being any one of the multiple augmented reality devices, the method comprising: responding to a consultation request acting on the first augmented reality device, obtaining first three-dimensional point cloud data corresponding to the first augmented reality device; transmitting the first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to a second augmented reality device via the cloud, wherein the second augmented reality device is an augmented reality device other than the first augmented reality device in the multidisciplinary consultation system; and obtaining second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud; rendering the second three-dimensional point cloud data to obtain a three-dimensional picture corresponding to the second augmented reality device, and displaying the three-dimensional picture.
[0007] In combination with the first aspect, an embodiment of the present invention provides a first implementation of the first aspect, wherein the step of obtaining first three-dimensional point cloud data corresponding to the first augmented reality device includes: starting a three-dimensional scanning device configured in the first augmented reality device; scanning the body surface data of the target user through the three-dimensional scanning device; and generating first three-dimensional point cloud data based on the body surface data.
[0008] In combination with the first aspect, an embodiment of the present invention provides a second implementation of the first aspect, wherein the first augmented reality device is configured with an interactive interface, and the interactive interface is configured with multiple functional controls; the method also includes: responding to the target user acting on the selection control included in the functional control, obtaining user information corresponding to the target user; determining whether the target user corresponding to the user information has operation authority; if so, generating a consultation request.
[0009] In combination with the first aspect, an embodiment of the present invention provides a third implementation of the first aspect, wherein the function control includes a meeting application control; the method also includes: responding to the selection operation of the target user on the meeting application control; generating a meeting time based on the selection operation; transmitting the meeting time to the cloud, so as to transmit the meeting time to the second augmented reality device through the cloud.
[0010] In combination with the first aspect, an embodiment of the present invention provides a fourth implementation of the first aspect, wherein the method also includes: generating a corresponding audio signal in response to voice input information output by the first augmented reality device; transmitting the audio signal to the cloud, so as to transmit the audio signal to the second augmented reality device through the cloud.
[0011] In combination with the first aspect, an embodiment of the present invention provides a fifth implementation of the first aspect, wherein the method further includes: performing data missing detection on the rendered second three-dimensional point cloud data; if data missing is detected in the second three-dimensional point cloud data, performing linear interpolation processing on the missing data.
[0012] In combination with the first aspect, an embodiment of the present invention provides a sixth implementation of the first aspect, wherein the step of obtaining first three-dimensional point cloud data corresponding to the first augmented reality device further includes: scanning the inspection report of the target user through a three-dimensional scanning device configured with the first augmented reality device; and generating first three-dimensional point cloud data based on the inspection report.
[0013] In combination with the first aspect, an embodiment of the present invention provides a seventh implementation of the first aspect, wherein the examination report includes medical imaging data of the target user; the step of generating first three-dimensional point cloud data according to the examination report includes: performing organ segmentation and modeling processing on the medical imaging data to generate first three-dimensional point cloud data.
[0014] In a second aspect, an embodiment of the present invention provides a multidisciplinary consultation device based on real-time point cloud rendering, wherein the device is applied to the above-mentioned method; the device includes: a data acquisition module, used to respond to a consultation request acting on a first augmented reality device, and obtain first three-dimensional point cloud data corresponding to the first augmented reality device; a data transmission module, used to transmit the first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to a second augmented reality device through the cloud, wherein the second augmented reality device is an augmented reality device other than the first augmented reality device in the multidisciplinary consultation system; and, obtaining second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud; a data processing module, used to render the second three-dimensional point cloud data, obtain a three-dimensional picture corresponding to the second augmented reality device, and display the three-dimensional picture.
[0015] In a third aspect, an embodiment of the present invention provides a multidisciplinary consultation system based on real-time point cloud rendering, wherein the system is configured with the above-mentioned device and is used to execute the above-mentioned method.
[0016] The embodiments of the present invention bring the following beneficial effects: The present invention provides a multidisciplinary consultation method, device and system based on real-time point cloud rendering, the method is applied to the first augmented reality device of the multidisciplinary consultation system, the multidisciplinary consultation system includes multiple augmented reality devices interconnected through the cloud, and the first augmented reality device is any one of them. After responding to the consultation request acting on the first augmented reality device, the present invention transmits the acquired first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to the second augmented reality device through the cloud, and obtains the second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud, and renders the second three-dimensional point cloud data to obtain a three-dimensional picture, and displays the three-dimensional picture. Based on this, multiple experts can understand the patient's information more realistically and comprehensively, so as to conduct full and in-depth discussions and exchanges of opinions.
[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 A flowchart of a multidisciplinary consultation method based on real-time point cloud rendering provided by an embodiment of the present invention;
[0021] Figure 2 A flowchart of another multidisciplinary consultation method based on real-time point cloud rendering provided by an embodiment of the present invention;
[0022] Figure 3 A logical diagram of a virtual consultation space provided by an embodiment of the present invention;
[0023] Figure 4 A schematic diagram of a format of three-dimensional point cloud data provided by an embodiment of the present invention;
[0024] Figure 5A schematic diagram of the structure of a multidisciplinary consultation device based on real-time point cloud rendering provided by an embodiment of the present invention;
[0025] Figure 6 A schematic diagram of the structure of another multidisciplinary consultation device based on real-time point cloud rendering provided by an embodiment of the present invention;
[0026] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] The present invention provides a multidisciplinary consultation method, device and system based on real-time point cloud rendering, so that multiple experts can understand the patient's information more realistically and comprehensively, thereby conducting full and in-depth discussions and exchanges of opinions.
[0029] To facilitate understanding of this embodiment, a multidisciplinary consultation method based on real-time point cloud rendering disclosed in an embodiment of the present invention is first introduced in detail, wherein the method is applied to a first augmented reality device of a multidisciplinary consultation system, the multidisciplinary consultation system includes multiple augmented reality devices, and the multiple augmented reality devices are interconnected through the cloud, and the first augmented reality device is any one of the multiple augmented reality devices. Figure 1 A schematic diagram of a multidisciplinary consultation method based on real-time point cloud rendering provided by an embodiment of the present invention is shown. Figure 1 , the method comprises the following steps:
[0030] Step S102, in response to a consultation request acting on a first augmented reality device, obtaining first three-dimensional point cloud data corresponding to the first augmented reality device.
[0031] Step S104: transmitting the first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to the second augmented reality device through the cloud; and obtaining the second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud.
[0032] Step S106, rendering the second three-dimensional point cloud data to obtain a three-dimensional image corresponding to the second augmented reality device, and displaying the three-dimensional image.
[0033] In a specific implementation, the second augmented reality device is an augmented reality device other than the first augmented reality device in the multidisciplinary consultation system. In the embodiment of the present invention, the augmented reality devices of multiple experts communicate with the cloud respectively, the first augmented reality device is the device used by the user, and the second augmented reality device is the device used by other experts.
[0034] The patient's MDT application can be submitted by any expert. After the review center approves the current application, the experts of the consultation team will bring augmented reality devices at the agreed time and enter the same virtual consultation space. The doctor's augmented reality device will also join the virtual room created in the cloud, that is, the virtual local area network in the same network space. The devices in the network can synchronously upload / download the point cloud data of the server and render the data. Among them, the augmented reality device provides the corresponding three-dimensional point cloud data to the cloud. Further, after rendering the three-dimensional point cloud data into a three-dimensional picture, the doctor can see the real-time three-dimensional picture of other participating experts in their respective devices, communicate by voice, and virtually interact with the patient's examination report. Through this three-dimensional real-time rendering method, the experts participating in the MDT consultation can participate in the discussion more realistically remotely, have a more comprehensive understanding of the patient's information through three-dimensional rendering, and can conduct full and in-depth discussions and exchanges of opinions, which is helpful for obtaining a comprehensive diagnosis result.
[0035] Furthermore, based on the above embodiment, the embodiment of the present invention also provides another multidisciplinary consultation method based on real-time point cloud rendering. Figure 2 A flowchart of another multidisciplinary consultation method based on real-time point cloud rendering provided by an embodiment of the present invention is shown. Figure 2 , the method comprises the following steps:
[0036] Step S202: responding to a consultation request on the first augmented reality device.
[0037] Among them, the first augmented reality device is configured with an interactive interface, and the interactive interface is configured with multiple functional controls; the embodiment of the present invention obtains user information corresponding to the target user by responding to the target user acting on the selection control included in the functional control; determines whether the target user corresponding to the user information has operation authority; if so, generates a consultation request.
[0038] In specific implementation, users can apply for an MDT meeting by operating the interactive interface of their respective augmented reality devices and selecting the functional controls of the interactive interface. The review center of the multidisciplinary consultation system reviews the selection of the functional controls to determine whether the user has the authority to participate in or apply for the meeting. If the review is passed, other experts are organized to enter the virtual consultation space composed of the interconnected augmented reality devices. Figure 3A logical diagram of a virtual consultation space provided by an embodiment of the present invention is shown.
[0039] In a specific implementation, the functional control includes a meeting application control; the embodiment of the present invention generates a meeting time by responding to the target user's selection operation on the meeting application control; the meeting time is transmitted to the cloud, so that the meeting time is transmitted to the second augmented reality device through the cloud, thereby organizing the time of the expert group for joint consultation. In one embodiment, information such as the meeting time and location can be notified to the participants through electronic terminals such as the user's mobile phone.
[0040] Step S204: start the three-dimensional scanning device configured in the first augmented reality device.
[0041] Step S206: Scan the body surface data of the target user by using a three-dimensional scanning device.
[0042] Step S208: Generate first three-dimensional point cloud data according to the body surface data.
[0043] A 3D scanner is placed in each expert's independent space. This device scans the expert and patient's body surface data and generates a 3D point cloud data set P (x, y, z, r, g, b). Each point cloud data contains both the coordinate position relative to the 3D scanner and the real color information. These point cloud data are uploaded to the cloud in a specified format and then transmitted to the augmented reality devices of different experts for real-time 3D point cloud rendering. Figure 4 ,The custom format of the multidisciplinary consultation system contains patient information, location and color information. These format data are transmitted to different terminals through the network to render models and display text information.
[0044] Furthermore, the examination report of the target user can be scanned by a three-dimensional scanning device configured by the first augmented reality device; and the first three-dimensional point cloud data can be generated according to the examination report. In addition, the examination report includes the medical image data of the target user, and the medical image data can also be processed by organ segmentation and modeling to generate the first three-dimensional point cloud data.
[0045] In actual implementation, the patient's medical history and paper examination reports can also be scanned into 3D point cloud data for real-time rendering. Furthermore, the patient's CT / MRI medical imaging data can be segmented and modeled into 3D data, which can also be rendered and viewed on the doctor's device.
[0046] Step S210: transmitting the first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to the second augmented reality device through the cloud; and obtaining the second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud.
[0047] Step S212: Render the second three-dimensional point cloud data to obtain a three-dimensional image corresponding to the second augmented reality device, and display the three-dimensional image.
[0048] Furthermore, the embodiment of the present invention also performs data missing detection on the rendered second three-dimensional point cloud data; if data missing is detected in the second three-dimensional point cloud data, linear interpolation is performed on the missing data. In specific implementation, if a certain segment of data is lost as a whole or part of the data is missing during data transmission, in order to ensure the real-time display of the current frame, the embodiment of the present invention also performs linear interpolation on the missing data, including bilinear interpolation, cubic spline interpolation, etc. Completing the missing data ensures the real-time performance of the rendering.
[0049] Furthermore, multiple experts can virtually interact with each other in a virtual consultation space to review the patient's examination report based on the augmented reality device. In addition, real-time voice interactive communication can be carried out to remotely discuss the patient's surgical plan information. The embodiment of the present invention is also used to respond to the voice input information output by the first augmented reality device, generate a corresponding audio signal, and transmit the audio signal to the cloud, so as to transmit the audio signal to the second augmented reality device through the cloud.
[0050] In summary, the embodiment of the present invention can also create a clinical knowledge base for consultation plans, integrate consultation reports into the medical record system, save the results of expert discussions, and form electronic medical record files, including the patient's pathology and imaging examination reports, treatment plans and other information.
[0051] Furthermore, based on the above embodiments, the embodiments of the present invention also provide a multidisciplinary consultation device based on real-time point cloud rendering. Figure 5 FIG. 4 shows a schematic diagram of a multidisciplinary consultation device based on real-time point cloud rendering provided by an embodiment of the present invention. Figure 5 As shown, the apparatus includes: a data acquisition module 100, for responding to a consultation request acting on a first augmented reality device, and acquiring first three-dimensional point cloud data corresponding to the first augmented reality device; a data transmission module 200, for transmitting the first three-dimensional point cloud data to a cloud, so as to transmit the first three-dimensional point cloud data to a second augmented reality device through the cloud, wherein the second augmented reality device is an augmented reality device other than the first augmented reality device in a multidisciplinary consultation system; and acquiring second three-dimensional point cloud data transmitted by the second augmented reality device from the cloud; a data processing module 300, for rendering the second three-dimensional point cloud data, obtaining a three-dimensional picture corresponding to the second augmented reality device, and displaying the three-dimensional picture.
[0052] An embodiment of the present invention provides a multidisciplinary consultation device based on real-time point cloud rendering, and its implementation principle and technical effects are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0053] Furthermore, based on the above embodiment, the embodiment of the present invention also provides another multidisciplinary consultation device based on real-time point cloud rendering. Figure 6 A structural schematic diagram of another multidisciplinary consultation device based on real-time point cloud rendering provided by an embodiment of the present invention is shown, wherein the above-mentioned data acquisition module 100 is also used to start a three-dimensional scanning device configured by a first augmented reality device; scan the body surface data of the target user through the three-dimensional scanning device; and generate first three-dimensional point cloud data based on the body surface data.
[0054] Among them, the first augmented reality device is configured with an interactive interface, and the interactive interface is configured with multiple functional controls; the device also includes a response module 400, which is used to respond to the target user's action on the selection control included in the functional control, and obtain user information corresponding to the target user; determine whether the target user corresponding to the user information has operation authority; if so, generate a consultation request.
[0055] Among them, the functional controls include a meeting application control; the above-mentioned response module 400 is also used to respond to the target user's selection operation on the meeting application control; based on the selection operation, generate a meeting time; transmit the meeting time to the cloud, so as to transmit the meeting time to the second augmented reality device through the cloud.
[0056] The data transmission module 200 is also used to respond to the voice input information output by the first augmented reality device, generate a corresponding audio signal, and transmit the audio signal to the cloud, so as to transmit the audio signal to the second augmented reality device through the cloud.
[0057] The data processing module 300 is further used to perform data missing detection on the rendered second three-dimensional point cloud data; if data missing is detected in the second three-dimensional point cloud data, linear interpolation processing is performed on the missing data.
[0058] The data acquisition module 100 is further configured to scan the inspection report of the target user through a three-dimensional scanning device configured by the first augmented reality device; and generate first three-dimensional point cloud data according to the inspection report.
[0059] The examination report includes the medical image data of the target user; the data acquisition module 100 is also used to perform organ segmentation and modeling processing on the medical image data to generate the first three-dimensional point cloud data.
[0060] Furthermore, based on the above embodiment, an embodiment of the present invention also provides a multidisciplinary consultation system based on real-time point cloud rendering, wherein the system is configured with the above device for executing the above method.
[0061] An embodiment of the present invention provides a multidisciplinary consultation system based on real-time point cloud rendering, and its implementation principle and technical effects are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0062] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above Figure 1 to Figure 2 Any of the steps of the method shown.
[0063] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor. Figure 1 to Figure 2 The embodiment of the present invention also provides a structural diagram of an electronic device, such as Figure 7 FIG. 1 is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 71 and a memory 70, the memory 70 stores computer executable instructions that can be executed by the processor 71, and the processor 71 executes the computer executable instructions to implement the above Figure 1 to Figure 2 Any of the methods shown. Figure 7 In the illustrated embodiment, the electronic device further includes a bus 72 and a communication interface 73 , wherein the processor 71 , the communication interface 73 and the memory 70 are connected via the bus 72 .
[0064] Among them, the memory 70 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 72 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc., and can also be an AMBA (Advanced Microcontroller Bus Architecture, on-chip bus standard) bus, wherein AMBA defines three types of buses, including APB (Advanced Peripheral Bus) bus, AHB (Advanced High-performance Bus) bus and AXI (Advanced eXtensible Interface) bus. The bus 72 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0065] The processor 71 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 71. The above processor 71 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 71 reads the information in the memory and combines its hardware to complete the above Figure 1 to Figure 2 Any of the methods shown.
[0066] A computer program product of a multidisciplinary consultation method, device and system based on real-time point cloud rendering provided by an embodiment of the present invention includes a computer-readable storage medium storing a program code, and the instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can refer to the method embodiment, which will not be repeated here. A person skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. In addition, in the description of the embodiment of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood in specific circumstances. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.
[0067] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A multidisciplinary consultation method based on real-time point cloud rendering, characterized in that: The method is applied to a first augmented reality device of a multidisciplinary consultation system, the multidisciplinary consultation system includes multiple augmented reality devices, and the multiple augmented reality devices are interconnected via a cloud, the first augmented reality device is any one of the multiple augmented reality devices, and the method includes: In response to a consultation request acting on the first augmented reality device, obtaining first three-dimensional point cloud data corresponding to the first augmented reality device; Transmitting the first three-dimensional point cloud data to the cloud, so as to transmit the first three-dimensional point cloud data to a second augmented reality device through the cloud, wherein the second augmented reality device is another augmented reality device in the multidisciplinary consultation system except the first augmented reality device; and, obtaining, from the cloud, second three-dimensional point cloud data transmitted by the second augmented reality device; The second three-dimensional point cloud data is rendered to obtain a three-dimensional image corresponding to the second augmented reality device, and the three-dimensional image is displayed.
2. The method according to claim 1, characterized in that The step of acquiring first three-dimensional point cloud data corresponding to the first augmented reality device includes: Starting the three-dimensional scanning device configured by the first augmented reality device; Scanning the body surface data of the target user by means of the three-dimensional scanning device; The first three-dimensional point cloud data is generated according to the body surface data.
3. The method according to claim 1, characterized in that The first augmented reality device is configured with an interactive interface, and the interactive interface is configured with a plurality of functional controls; The method further comprises: In response to a target user acting on a selection control included in the function control, obtaining user information corresponding to the target user; Determine whether the target user corresponding to the user information has operation authority; If yes, the consultation request is generated.
4. The method according to claim 3, characterized in that: The function control includes a conference application control; and the method further includes: Responding to a selection operation of the target user on the conference application control; Based on the selection operation, generate a meeting time; The meeting time is transmitted to the cloud, so that the meeting time is transmitted to the second augmented reality device through the cloud.
5. The method according to claim 1, characterized in that The method further comprises: In response to the voice input information output by the first augmented reality device, generate a corresponding audio signal; The audio signal is transmitted to the cloud, so that the audio signal is transmitted to a second augmented reality device through the cloud.
6. The method according to claim 1, characterized in that The method further comprises: Performing data missing detection on the rendered second three-dimensional point cloud data; If it is detected that the second three-dimensional point cloud data is missing, linear interpolation processing is performed on the missing data.
7. The method according to claim 2, characterized in that The step of acquiring first three-dimensional point cloud data corresponding to the first augmented reality device also includes: Scanning the inspection report of the target user through the three-dimensional scanning device configured by the first augmented reality device; The first three-dimensional point cloud data is generated according to the inspection report.
8. The method according to claim 7, characterized in that The examination report includes medical imaging data of the target user; and the step of generating the first three-dimensional point cloud data according to the examination report includes: Organ segmentation and modeling are performed on the medical image data to generate first three-dimensional point cloud data.
9. A multidisciplinary consultation device based on real-time point cloud rendering, characterized in that: The device is applied to the method described in any one of claims 1 to 8; the device comprises: A data acquisition module, configured to respond to a consultation request acting on a first augmented reality device and acquire first three-dimensional point cloud data corresponding to the first augmented reality device; a data transmission module, configured to transmit the first three-dimensional point cloud data to a cloud, so as to transmit the first three-dimensional point cloud data to a second augmented reality device through the cloud, wherein the second augmented reality device is an augmented reality device other than the first augmented reality device in the multidisciplinary consultation system; and, obtaining, from the cloud, second three-dimensional point cloud data transmitted by the second augmented reality device; A data processing module is used to render the second three-dimensional point cloud data, obtain a three-dimensional picture corresponding to the second augmented reality device, and display the three-dimensional picture.
10. A multidisciplinary consultation system based on real-time point cloud rendering, characterized in that: The system is configured with the apparatus of claim 9, and is used to execute the method of any one of claims 1-8.
