Information processing system, server device, and client device

Abstract

To provide an information processing system, a server device, and a client device capable of reducing introduction costs and processing convenient three dimensional data.SOLUTION: The three dimensional scanner system 1 includes a scanner 200, a server device 100, and a client device 300. The scanner 200 scans an object to sequentially acquire three dimensional data of the object. When server 100 generates combined data representing a three dimensional shape of a target object by joining pieces of three dimensional data based on an overlapping portion in the pieces of three dimensional data sequentially transmitted from scanner 200, server 100 generates as difference data, a difference between pieces of combined data before and after joining of the pieces of three dimensional data. The client device 300 accumulates the differential images sequentially transmitted from the server device 100, and generates a two-dimension image of the object viewed from an arbitrary viewpoint based on the accumulated differential images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, a server device, and a client device. [Background technology]

[0002] In the field of dentistry, scanners have been developed to acquire the three-dimensional shape of teeth in order to digitally design prostheses and the like on a computer (see, for example, Patent Document 1: Japanese Patent No. 5654583). The scanner acquires the three-dimensional shape of the surface of an object using, for example, focusing techniques to generate three-dimensional data, and then calculates and displays a two-dimensional projection drawing based on the three-dimensional data. This requires a computing device connected to the scanner to perform high-load processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5654583 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when multiple scanners are introduced into a hospital, it is necessary to prepare an equal number of arithmetic units that perform high-load processing, which increases the introduction cost. Also, when one scanner is used in multiple locations within the hospital (each chair unit), it is necessary to move not only the scanner but also the arithmetic unit to the location where it is used, which may reduce convenience.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an information processing system, server device, and client device that can process three-dimensional data with reduced implementation costs and high convenience. [Means for solving the problem]

[0006] An information processing system according to the present disclosure is an information processing system for processing three-dimensional data. The information processing system includes a scanner, a server device, and a client device. The scanner scans an object to sequentially acquire three-dimensional data of the object. The server device stitches together the three-dimensional data based on overlapping portions of the three-dimensional data sequentially transmitted from the scanner to generate combined data representing the three-dimensional shape of the object. The server device generates difference data representing the difference between the combined data before and after stitching the three-dimensional data. The client device accumulates the difference data sequentially transmitted from the server device and generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data.

[0007] A server device according to the present disclosure is a server device that transmits and receives data to and from a client device included in an information processing system that processes three-dimensional data. The server device includes a receiving unit, a computing unit, and a transmitting unit. The receiving unit receives three-dimensional data sequentially transmitted from a scanner that scans an object and sequentially acquires three-dimensional data of the object. The computing unit generates difference data representing the difference between the combined data before and after the three-dimensional data is joined together when generating combined data representing the three-dimensional shape of the object by joining the three-dimensional data sequentially received by the receiving unit based on overlapping portions of the three-dimensional data. The transmitting unit accumulates the difference data generated by the computing unit and transmits the difference data to a client device that generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data.

[0008] A client device according to the present disclosure is a client device that transmits and receives data to and from a server device included in an information processing system that processes three-dimensional data. The client device includes a receiving unit, a computing unit, and a display unit. The receiving unit receives difference data transmitted from the server device, which generates difference data representing the difference between the combined data before and after joining the three-dimensional data when joining the three-dimensional data based on overlapping portions of the three-dimensional data sequentially transmitted from a scanner that scans the object and sequentially acquires three-dimensional data of the object to generate combined data representing the three-dimensional shape of the object. The computing unit accumulates the difference data received by the receiving unit and generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data. The display unit displays the two-dimensional image. [Effects of the Invention]

[0009] The information processing system according to the present disclosure includes a scanner, a server device, and a client device, thereby reducing implementation costs and providing a highly convenient information processing system. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing a configuration of a three-dimensional scanner system according to an embodiment; [Figure 2] FIG. 2 is a block diagram showing a configuration of a server device according to an embodiment. [Figure 3] FIG. 2 is a block diagram showing a configuration of a client device according to an embodiment. [Figure 4] FIG. 2 is a sequence diagram for explaining the processing of three-dimensional data in the three-dimensional scanner system according to the embodiment. [Figure 5] FIG. 2 is a schematic diagram for explaining an example of processing of three-dimensional data in the three-dimensional scanner system according to the embodiment. [Figure 6] FIG. 10 is a sequence diagram for explaining a deletion process in the three-dimensional scanner system according to the embodiment. [Figure 7]FIG. 10 is a schematic diagram for explaining an example of a deletion process in the three-dimensional scanner system according to the embodiment. [Figure 8] FIG. 10 is a sequence diagram for explaining an unnecessary object removal process in the three-dimensional scanner system according to the embodiment. [Figure 9] FIG. 10 is a schematic diagram for explaining an example of an unnecessary object removal process in the three-dimensional scanner system according to the embodiment. [Figure 10] FIG. 10 is a sequence diagram for explaining a positional deviation correction process in the three-dimensional scanner system according to the embodiment. [Figure 11] 10A and 10B are schematic diagrams for explaining an example of a positional deviation correction process in the three-dimensional scanner system according to the embodiment. [Figure 12] FIG. 10 is a sequence diagram for explaining margin line processing in the three-dimensional scanner system according to the embodiment. [Figure 13] FIG. 10 is a sequence diagram for explaining an example of a dental arch scanning process in the three-dimensional scanner system according to the embodiment. [Figure 14] FIG. 10 is a schematic diagram showing the configuration of a three-dimensional scanner system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] An information processing system according to an embodiment will be described with reference to the drawings. In the embodiment, a three-dimensional scanner system used in dental treatment will be described as one exemplary form of the information processing system. The three-dimensional scanner system is an intraoral scanner system for acquiring the three-dimensional shape of teeth in the oral cavity. The information processing system according to the embodiment is not limited to an intraoral scanner system, but can also be applied to other information processing systems having a similar configuration for processing three-dimensional data. For example, the information processing system can also be applied to an information processing system that processes three-dimensional data inside the outer ear by capturing images of the inside of a human ear in addition to the inside of the oral cavity.

[0012] The field in which the information processing system according to the embodiment is used is not limited to dentistry, but can be used in all medical fields such as ophthalmology, otolaryngology, radiology, and veterinary medicine, and can also be used in other fields other than medicine, such as industrial fields, etc. In addition, in this disclosure, the term "diagnosis" also includes the meaning of "diagnosis" and "treatment."

[0013] [Configuration of 3D scanner system] FIG. 1 is a schematic diagram showing the configuration of a three-dimensional scanner system 1 according to an embodiment. As shown in FIG. 1, the three-dimensional scanner system 1 includes a server device 100, a scanner 200, and a client device 300. The server device 100 is a computing device that performs high-load processing, such as stitching together three-dimensional data and generating mesh data of three-dimensional shapes. The server device 100 is installed in a location within the hospital that is different from the location where medical treatment is performed (for example, in the backroom of the hospital), or in a location outside the hospital. The server device 100 may also be a cloud server.

[0014] The scanner 200 inserts a probe portion into the oral cavity, projects light having a pattern onto an object such as a tooth, measures the light reflected from the object onto which the pattern is projected, and acquires data on the three-dimensional shape of the object using the principle of focusing. Note that the scanner 200 is not limited to a configuration that acquires data on the three-dimensional shape of the object using the principle of focusing, and may be configured using other principles as long as it is configured to acquire data on the three-dimensional shape of the object using an optical method such as the principle of confocal imaging.

[0015] The scanner 200 is connected to the cradle 210 via a wired cable, and transmits the acquired data of the three-dimensional shape of the object to the cradle 210 via the wired cable. The cradle 210 is network-connected to the server device 100 via a LAN cable 220. Specifically, the cradle 210 is network-connected to the server device 100 via a network hub 600 by inserting the LAN cable 220 into a socket 601 provided in each chair unit 400 as shown in FIG.

[0016] It is preferable that the network hub 600 has, for example, a PoE (Power over Ethernet) function. If the network hub 600 has the PoE function, it can supply power to the scanner 200 and the cradle 210 through the LAN cable 220. If power can be supplied to the scanner 200 and the cradle 210 through the LAN cable 220, it becomes unnecessary to supply power to the cradle 210 using an AC adapter or the like.

[0017] Power is supplied from the cradle 210 to the scanner 200 via a wired cable connecting the cradle 210 and the scanner 200. However, in a configuration in which the scanner 200 and the cradle 210 are connected wirelessly and data on the three-dimensional shape of an object acquired by the scanner 200 is transmitted to the cradle 210, the scanner 200 needs to have a built-in storage battery. Note that the built-in storage battery of the scanner 200 may be configured to be charged when the scanner 200 is placed on the cradle 210.

[0018] The client device 300 displays the three-dimensional shape data processed by the server device 100 on the display device 500. The client device 300 performs processes with a lower load than the server device 100, such as a process for displaying the three-dimensional shape data on the display device 500. Therefore, the performance specifications required of the client device 300 are lower than those required of the server device 100, and the price is also lower. For example, one server device 100 is installed in a hospital, while a client device 300 is installed in each chair unit 400.

[0019] In this way, by dividing the 3D scanner system 1 into the server device 100 and the client device 300, high-load processing is performed by the server device 100 and low-load processing is performed by the client device 300, enabling distribution of processing. Generally, computing devices with high performance specifications are large and heavy, making them difficult to move within a hospital. On the other hand, computing devices with low performance specifications are small and light, making them easy to move within a hospital. Therefore, the server device 100 is installed in a space separate from the examination room, such as a backroom within the hospital, and the client device 300 is placed on each chair unit 400, displaying the 3D shape of the object being scanned on the display device 500. In this way, the 3D scanner system 1 can reduce implementation costs and provide a highly convenient information processing system.

[0020] Scanner 200 can be scanned by inserting LAN cable 220 into a socket 601 provided on each chair unit 400 and moving between chair units 400. Specifically, scanner 200 and cradle 210 are placed on wagon 230 shown in Fig. 1, and wagon 230 is moved, and LAN cable 220 is inserted into socket 601 of the chair unit 400 that uses scanner 200. Note that an input device such as a mouse 652 may be placed on wagon 230, and the viewpoint of the image of the object displayed on display device 500 may be changed as desired using mouse 652.

[0021] The three-dimensional scanner system 1 is configured (backyard configuration) separately: a server device 100 that processes three-dimensional shape data; and a client device 300 that displays images of the three-dimensional shape data on a display device 500. However, instead of providing two computing devices, such as the server device 100 and the client device 300, the three-dimensional scanner system 1 may be configured (standalone configuration) in which the functions of the server device 100 and the functions of the client device 300 are built into a single computing device. Note that configuring the three-dimensional scanner system 1 in a standalone configuration eliminates the need for the seller to line up multiple computing devices, thereby reducing the effort and cost of system maintenance and management. Furthermore, when configuring the three-dimensional scanner system 1 in a standalone configuration, scanning of objects at the visit site is possible simply by bringing a computing device that has the functions of the server device 100 and the client device 300 built into it, and a scanner 200, when visiting a clinic. On the other hand, even if the three-dimensional scanner system 1 is configured in a backyard configuration, if it can be connected to a server device 100 installed in a hospital or on the cloud using an internet line, it will be possible to scan objects at the destination by simply bringing the client device 300 and the scanner 200 to the destination.

[0022] Furthermore, the three-dimensional scanner system 1 is not limited to a configuration in which an application program for processing three-dimensional shape data is installed in the server device 100 and an application program for displaying an image of the three-dimensional shape data on the display device 500 is installed in the client device 300. For example, the server device 100 may function as a web server, and a web application program that runs on a web browser may be installed in the client device 300. Installing a web application program rather than a dedicated application program in the client device 300 eliminates the need to install a dedicated application program and facilitates program upgrades. Furthermore, because the web application program runs on a browser, it can be used on various computing devices (e.g., desktop PCs, laptops, tablet devices, smartphones, etc.) and is not limited to a specific operating system (e.g., Windows®, Mac OS®, Linux®, Android®, Chrome OS®, etc.) (multi-platform compatibility).

[0023] The three-dimensional scanner system 1 does not utilize a remote desktop function that allows the sending and receiving of input information, such as the destination desktop screen and the source keyboard and mouse, via the Internet. In the three-dimensional scanner system 1, as described below, the server device 100 generates differential data, which is the difference between the combined data before and after stitching together the three-dimensional data, and transmits this differential data to the client device 300. Therefore, the three-dimensional scanner system 1 is less susceptible to performance degradation due to the speed of the Internet connection, such as an inability to display an image of the target object on the display device 500 or display delays. The differential data is point cloud data or mesh data composed of points indicating three-dimensional coordinate positions.

[0024] Furthermore, in the 3D scanner system 1, if the client device 300 breaks down and needs to be replaced with another computing device, it is only necessary for the web browser to work, and there is no need to reinstall the dedicated application program. The client device 300 only needs to have the performance specs to run the web browser, and there is no need for a computing device with high performance specs, which reduces the introduction cost.

[0025] Furthermore, even if the network is disconnected during scanning and the client device 300 cannot communicate with the server device 100, the 3D scanner system 1 can perform rendering processes on the client device 300, such as rotating, scaling, and moving the 3D shape of the object, based on the differential data stored in the client device 300, as described below. In addition, image management systems, application programs for providing explanations to patients, and application programs for sending and receiving data (3D data of the object, dental laboratory instructions, etc.) and messages between dental clinics and dental laboratories are often web application programs. Furthermore, because clinics often use the same network, web application programs can be easily switched by switching tabs in a web browser. Therefore, using web application programs to run on the client device 300 increases compatibility.

[0026] If a computing device is already installed in the chair unit 400, the computing device can be used as the client device 300. Furthermore, the display device installed in the chair unit 400 can be used as the display device 500. Displaying an image of the object being scanned on the display device installed in the chair unit 400 reduces the need for the surgeon (user) to move their line of sight. Furthermore, the scanner 200 does not need to be placed on the cart 230 but can be placed on a tray in the chair unit 400. In this case, the cart 230 is unnecessary, and space in the examination room is not consumed. Furthermore, by using a Web application program as the application program to be run on the client device 300, the license dongle only needs to be connected to the server device 100, eliminating the need to provide a license dongle for each client device 300. In other words, when moving the scanner 200 to another chair unit 400, it is not necessary to move the license dongle from the currently connected client device 300 to the destination client device 300 and connect it thereto.

[0027] [Server device configuration] An example of the hardware configuration of the server device 100 according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the server device 100 according to the embodiment. The server device 100 may be realized, for example, by a general-purpose computer, or by a computer dedicated to the three-dimensional scanner system 1.

[0028] As shown in FIG. 2, the server device 100 includes, as its main hardware elements, a LAN interface 102, a display interface 103, a peripheral device interface 105, a wireless communication interface 106, a media reading device 107, a memory 109, a storage 110, and an arithmetic circuit 130.

[0029] The LAN interface 102 is network-connected to the scanner 200, the client device 300, other locally located computing devices within the hospital, the Internet, and the like, via the network hub 600 shown in Fig. 1. Note that Fig. 2 shows the scanner 200 as being directly connected to the LAN interface 102. The LAN interface 102 is an interface for connecting the scanner 200, and also realizes data input / output between the server device 100 and the client device 300, other locally located computing devices, cloud servers, and the like.

[0030] The display interface 103 is an interface for connecting a display device 140 for the server device 100, and realizes input and output of data between the server device 100 and the display device 140. The display device 140 is configured, for example, with an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display.

[0031] The peripheral device interface 105 is an interface for connecting peripheral devices such as a keyboard 160 and a mouse 161 for the server device 100, and realizes input and output of data between the server device 100 and the peripheral devices.

[0032] The wireless communication interface 106 uses wireless communication to transmit and receive data between devices located within the hospital, other locally located computing devices, cloud servers, etc. The wireless communication interface 106 supports any communication method, such as mobile communication (4G, 5G, etc.), wireless LAN (Local Area Network), Bluetooth (registered trademark), etc.

[0033] The media reader 107 reads various data and programs stored on the removable disk 150 .

[0034] The memory 109 provides a storage area for temporarily storing program code, work memory, etc. when the arithmetic circuit 130 executes any program. The memory 109 is configured by a volatile memory device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

[0035] The storage 110 provides a storage area for storing various data such as three-dimensional data of an object and programs. The storage 110 is configured, for example, by a nonvolatile memory device such as a hard disk or an SSD (Solid State Drive).

[0036] The storage 110 stores scan data 112, point cloud data 114, display mesh data 116, combined data 118, difference data 119, saved mesh data 120, deleted data 122, a scan program 124, a web server program 126, and an OS (Operating System) 127.

[0037] The scan data 112 is three-dimensional data acquired by the scanner 200. The point cloud data 114 is data obtained by extracting point cloud information of the object from the scan data 112. The display mesh data 116 is mesh data generated from the scan data 112 obtained at the start of scanning, and is data to be combined with the scan data 112 subsequently transmitted in sequence from the scanner 200. The combined data 118 is data obtained by combining the display mesh data 116 with the scan data 112 subsequently transmitted in sequence from the scanner 200.

[0038] The differential data 119 is the difference between the combined data before and after stitching together the scan data 112. The mesh data for storage 120 is mesh data reconstructed using all of the scan data 112 acquired by the scanner 200 after scanning is completed. The deletion data 122 is data that is set in advance to identify unnecessary objects (e.g., fingers, medical instruments, etc.) that appear during scanning of the object. The scan program 124 is a program for processing three-dimensional data on the server device 100 during and after scanning of the object. The web server program 126 is a program for causing the server device 100 to function as a web server.

[0039] The arithmetic circuit 130 is an example of a computer, and is a computing entity that executes various programs to perform various processes in the server device 100. The arithmetic circuit 130 is configured with, for example, a central processing unit (CPU) 132, a field-programmable gate array (FPGA) 134, and a graphics processing unit (GPU) 136.

[0040] [Client device configuration] An example of the hardware configuration of client device 300 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of client device 300 according to this embodiment. Client device 300 may be realized, for example, by a general-purpose computer, or may be realized by a dedicated computer connected to chair unit 400.

[0041] As shown in FIG. 3, the client device 300 includes, as its main hardware elements, a display interface 301, a peripheral device interface 303, a LAN interface 305, a wireless communication interface 306, a media reading device 307, a memory 309, a storage 310, and an arithmetic circuit 330.

[0042] The display interface 301 is an interface for connecting the display device 500, and realizes input and output of data between the client device 300 and the display device 500. The display device 500 is configured, for example, with an LCD or an organic EL display.

[0043] The peripheral device interface 303 is an interface for connecting peripheral devices such as a keyboard 651 and a mouse 652, and realizes input and output of data between the client device 300 and the peripheral devices.

[0044] The LAN interface 305 is connected to the server device 100, other locally located computing devices within the hospital, the Internet, and the like via a network hub 600 shown in Fig. 1. Note that Fig. 3 shows the server device 100 as being directly connected to the LAN interface 305. The LAN interface 305 realizes input and output of data between the server device 100 and the client device 300, other locally located computing devices, and cloud servers, etc.

[0045] The wireless communication interface 306 uses wireless communication to transmit and receive data between devices located within the hospital, other locally located computing devices, cloud servers, etc. The wireless communication interface 306 supports any communication method, such as mobile communication (4G, 5G, etc.), wireless LAN, Bluetooth (registered trademark), etc.

[0046] The media reader 307 reads various data such as scan information stored on the removable disk 350 .

[0047] The memory 309 provides a storage area for temporarily storing program code, work memory, etc. when the arithmetic circuit 330 executes any program. The memory 309 is configured by a volatile memory device such as a DRAM or an SRAM, for example.

[0048] Storage 310 provides a memory area for storing various data and programs required for processing such as displaying an image of an object during and after scanning on display device 500. Storage 310 is configured, for example, by a non-volatile memory device such as a hard disk or SSD.

[0049] The storage 310 stores the display mesh data 116 , the difference data 119 , a two-dimensional image 314 , a web application program 321 , and an OS 327 .

[0050] The display mesh data 116 is mesh data generated from the scan data 112 obtained at the start of scanning, and is data transmitted from the server device 100 after scanning begins. The difference data 119 is the difference between the combined data before and after stitching together the scan data 112, and is data transmitted sequentially from the server device 100 during scanning. The two-dimensional image 314 is an image of the object viewed from an arbitrary viewpoint based on the accumulated difference data 119. The web application program 321 is an application program that runs on a web browser and displays the two-dimensional image 314 of the object on the display device 500.

[0051] The arithmetic circuit 330 is an example of a computer, and is an arithmetic entity that executes various programs to perform various processes, such as displaying a two-dimensional image 314 of an object based on the three-dimensional shape data processed by the server device 100 on the display device 500. The arithmetic circuit 330 is configured with, for example, a CPU 332, an FPGA 334, and a GPU 336.

[0052] [3D Scanner System Processing] Next, the processing of the three-dimensional scanner system 1 will be described using a sequence diagram. Fig. 4 is a sequence diagram for explaining the processing of three-dimensional data in the three-dimensional scanner system 1 according to the embodiment. First, upon receiving an operation to start scanning from the operator, the scanner 200 starts acquiring scan data 112 (three-dimensional data) of an object in the oral cavity (S201). The object is, for example, a dental arch in the oral cavity or a dental arch model.

[0053] The scanner 200 scans the dental arch in the oral cavity to sequentially acquire scan data 112 of the dental arch and sequentially transmits the acquired scan data 112 to the server device 100. The server device 100 converts the scan data 112 sequentially transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S101). The point cloud data 114 includes position information (coordinates of each axis in the vertical, horizontal, and height directions) of each point indicating the surface shape of the dental arch and color data indicating the color of each point. When converting the scan data 112 into point cloud data 114, the server device 100 may also perform a process of removing noise points from the point cloud data 114 and a process of smoothing the point cloud data 114. The server device 100 may also perform image recognition processing on each portion of the acquired point cloud data 114 to recognize which part of the dental arch it corresponds to.

[0054] The server device 100 generates display mesh data 116 corresponding to the point cloud data 114 (S102). The display mesh data 116 is the initial combined data obtained by sequentially stitching together scan data. Specifically, as shown in FIG. 5, the server device 100 generates the display mesh data 116 from scan data 112 obtained at the start of scanning. FIG. 5 is a schematic diagram for explaining an example of three-dimensional data processing in the three-dimensional scanner system 1 according to the embodiment. When a dental arch is scanned with the scanner 200, multiple scan data 112 are obtained at approximately the same position. To generate the display mesh data 116, not all of the multiple acquired scan data 112 are used, but some of the scan data 112 (for example, the first acquired scan data 112) are used. Therefore, the load on the server device 100 that generates the display mesh data 116 is low, and the display mesh data 116 can be generated in real time.

[0055] The server device 100 transmits the generated display mesh data 116 to the client device 300. The client device 300 generates a two-dimensional image 314 of the scanned dental arch viewed from an arbitrary viewpoint based on the received display mesh data 116 (S301). The client device 300 generates the two-dimensional image 314 of the scanned dental arch viewed from a viewpoint input by the surgeon using, for example, the mouse 652. The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S302). Specifically, the client device 300 calculates a two-dimensional projection of the display mesh data 116 viewed from the arbitrary viewpoint, and displays the obtained two-dimensional projection on the display device 500 as the two-dimensional image 314. For example, an algorithm such as perspective projection or parallel projection can be used to calculate the two-dimensional projection.

[0056] Next, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S103). Specifically, as shown in FIG. 5, the server device 100 receives scan data A and converts the scan data A into point cloud data 114. The server device 100 generates combined data 118 by splicing together the display mesh data 116 and the newly acquired scan data 112 based on the overlapping portions of the two pieces of data (S104). Specifically, as shown in FIG. 5, the server device 100 generates combined data A by splicing together the scan data A and the display mesh data based on the overlapping portions of the display mesh data and the scan data A. Note that the combined data 118 is generated using only some of the scan data 112 (for example, the first acquired scan data 112) rather than using all of the acquired scan data 112. Therefore, the load on the server device 100 that generates the combined data 118 is low, and the combined data 118 can be generated in real time.

[0057] When generating the combined data 118, the server device 100 generates the difference between the combined data 118 before and after splicing the scan data 112 as difference data 119, and transmits the difference data 119 to the client device 300. Specifically, the server device 100 generates the newly spliced ​​portion of the combined data A as difference data A, as shown in FIG. 5, and transmits the difference data A to the client device 300.

[0058] The client device 300 accumulates the difference data 119 transmitted from the server device 100 and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S303). Specifically, the client device 300 generates combined data A by connecting the difference data A to mesh data for display as shown in FIG. 5, and generates a two-dimensional image 314 of the scanned dental arch viewed from a viewpoint input by the surgeon using the mouse 652. The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S304). Specifically, the client device 300 calculates a two-dimensional projection of the combined data 118 viewed from an arbitrary viewpoint and displays the obtained two-dimensional projection on the display device 500 as the two-dimensional image 314. For example, an algorithm such as perspective projection or parallel projection can be used to calculate the two-dimensional projection. Since the client device 300 does not receive the combined data A from the server device 100 but only receives the differential data A, the amount of data transmitted from the server device 100 can be reduced.

[0059] Furthermore, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S110). Specifically, as shown in FIG. 5, the server device 100 receives scan data B and converts the scan data B into point cloud data 114. The server device 100 splices together the combined data 118 and the newly acquired scan data 112 based on the overlapping portions of the two data to generate new combined data 118 (S111). Specifically, the server device 100 generates combined data B by splicing together the scan data B with the combined data A based on the overlapping portions of the combined data A and the scan data B, as shown in FIG. 5.

[0060] When generating the combined data 118, the server device 100 generates the difference between the combined data 118 before and after splicing the scan data 112 as difference data 119, and transmits the difference data 119 to the client device 300. Specifically, the server device 100 generates the newly spliced ​​portion of the combined data B as difference data B, as shown in FIG. 5, and transmits the difference data B to the client device 300.

[0061] The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S310). Specifically, the client device 300 creates combined data B by combining difference data A and difference data B with display mesh data as shown in Fig. 5, and generates a two-dimensional image 314 of the scanned dental arch viewed from a viewpoint input by the surgeon using the mouse 652. The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S311).

[0062] Next, upon receiving an operation from the operator to end the scan, the scanner 200 ends acquisition of scan data 112 (three-dimensional data) of the intraoral object (S202). The scanner 200 transmits new scan data 112 acquired from the previous transmission of scan data 112 to the server device 100 until the end of the scan to the server device 100. The server device 100 converts the newly transmitted scan data 112 from the scanner 200 into point cloud data 114 (point cloud data conversion, S112). The server device 100 generates new combined data 118 by joining the two pieces of data based on the overlapping portions between the combined data 118 and the newly acquired scan data 112 (S113).

[0063] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S312). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S313).

[0064] After completing the scan, the server device 100 generates mesh data 120 for storage using all of the scan data 112 acquired by the scanner 200 (S114). When generating mesh data 120 for storage, the process of converting all of the scan data 112 into point cloud data 114 may also include processes such as removing noise points from the point cloud data 114, smoothing the point cloud data 114, and aligning all of the point cloud data 114. For example, after completing the scan, the server device 100 may minimize the stitching error of all of the scan data 112 using a pose graph optimization method. This allows the server device 100 to obtain three-dimensional dental arch data with the accuracy required for digitally designing prostheses and the like on a computer. The server device 100 may also align the entire point cloud data 114 of the upper and lower dental arches and transmit the alignment information to the client device 300.

[0065] The server device 100 stores the generated storage mesh data 120 in the storage 110 (S115). When the server device 100 generates the storage mesh data 120, it transmits the storage mesh data 120 to the client device 300. The client device 300 accumulates the storage mesh data 120 transmitted from the server device 100 and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated storage mesh data 120 (S314). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S315). The server device 100 may transmit the storage mesh data 120 stored in the storage 110 to an external device such as a cloud server, as necessary.

[0066] [Processing changes during scanning] Next, the change processing performed during scanning in the three-dimensional scanner system 1 will be explained using a sequence diagram. First, the deletion processing, which is one of the change processing, will be explained. FIG. 6 is a sequence diagram for explaining the deletion processing in the three-dimensional scanner system 1 according to the embodiment. FIG. 7 is a schematic diagram for explaining an example of the deletion processing in the three-dimensional scanner system 1 according to the embodiment. Here, "deletion" can include not only the complete deletion of data, but also the transfer of data to a storage area separate from the recording area in which the three-dimensional data of the object is stored. In other words, "deletion" means deleting or moving data from the storage area in which the three-dimensional data of the object is stored.

[0067] The server device 100 converts the newly transmitted scan data 112 from the scanner 200 into point cloud data 114 (point cloud data conversion, S120). The server device 100 generates new combined data 118 by splicing together the combined data 118 and the newly acquired scan data 112 based on the overlapping portions between the two data (S121). Specifically, the server device 100 generates combined data C by splicing together the scan data and the combined data based on the overlapping portions between the combined data and the scan data, as shown in FIG. 7.

[0068] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S320). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S321).

[0069] However, if the surgeon viewing the two-dimensional image 314 displayed on the display device 500 wishes to delete a portion of the combined data 118 for reasons such as rescanning, the surgeon can use the mouse 652 to specify the deletion range of the combined data 118. The client device 300 accepts the deletion range of the combined data 118 specified by the surgeon using the mouse 652 (S322). Specifically, the client device 300 allows the surgeon to specify the deletion range within the combined data C as shown in FIG. 7. Furthermore, the client device 300 transmits the specified deletion range of the combined data 118 to the server device 100.

[0070] The server device 100 identifies the data to be deleted from the combined data 118 based on the deletion range transmitted from the client device 300 (S122). The server device 100 changes the combined data 118 to the deleted data of the identified range (S123). Specifically, as shown in FIG. 7, the server device 100 changes the combined data C to combined data D by deleting the deletion range. Furthermore, the server device 100 generates difference data 119 between the combined data 118 before and after the deletion process, and transmits the difference data 119 to the client device 300. Note that the server device 100 may transmit to the client device 300 information about the faces and points of the deleted portion (information about the deleted portion) without generating difference data 119 between the combined data 118 before and after the deletion process. In this way, the server device 100 can reduce the amount of data transmitted to the client device 300 by transmitting only information about the deleted portion to the client device 300. Here, the information about the deleted portion includes information such as the vertex numbers of the mesh data and the face numbers of the mesh data.

[0071] The client device 300 updates the two-dimensional image 314 from which the deletion range has been deleted based on the difference data 119 generated by the deletion process (S323). When the client device 300 receives information about the deleted portion instead of the difference data 119, it updates the two-dimensional image 314 from which the deletion range has been deleted based on the information about the deleted portion. The client device 300 displays the updated two-dimensional image 314 on the display device 500 (S324).

[0072] Furthermore, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S124). The server device 100 connects the two pieces of data based on the overlapping portions of the combined data 118 after the deletion process and the newly acquired scan data 112, thereby generating new combined data 118 (S125).

[0073] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S325). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S326).

[0074] Next, an unnecessary object removal process, which is one of the change processes, will be described. Fig. 8 is a sequence diagram for explaining the unnecessary object removal process in the three-dimensional scanner system 1 according to the embodiment. Fig. 9 is a schematic diagram for explaining an example of the unnecessary object removal process in the three-dimensional scanner system 1 according to the embodiment.

[0075] The server device 100 converts the newly transmitted scan data 112 from the scanner 200 into point cloud data 114 (point cloud data conversion, S130). The server device 100 generates new combined data 118 by splicing together the combined data 118 and the newly acquired scan data 112 based on the overlapping portions between the two data (S131). Specifically, the server device 100 generates combined data E by splicing together the scan data and the combined data based on the overlapping portions between the combined data and the scan data, as shown in FIG. 9.

[0076] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S330). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S331).

[0077] However, during scanning of the dental arch, an unnecessary object (for example, a finger or a treatment tool) may appear in the two-dimensional image 314 displayed on the display device 500. Specifically, as shown in FIG. 9, an unnecessary object, a finger f, appears in the combined data F. If the surgeon wishes to delete the unnecessary object, the surgeon can set the client device 300 to automatically remove the unnecessary object. The client device 300 receives setting information for automatically removing the unnecessary object selected by the surgeon using the mouse 652 (S332). Note that the setting for automatically removing the unnecessary object can also be selected before starting scanning. Furthermore, the client device 300 transmits setting information for removing the unnecessary object to the server device 100.

[0078] Based on the setting information for removing unnecessary objects transmitted from the client device 300, the server device 100 identifies the unwanted object reflected in the combined data based on the preset unnecessary object removal data 122 (S132). The server device 100 changes the data of the identified unnecessary object to the deleted combined data 118 (S133). Specifically, the server device 100 changes the combined data F to the combined data Fa by deleting the finger f, as shown in FIG. 9. Furthermore, the server device 100 generates difference data 119, which is the difference between the combined data 118 before and after the unnecessary object removal process, and transmits the difference data 119 to the client device 300.

[0079] The client device 300 updates the two-dimensional image 314 based on the difference data 119 generated by the unnecessary object deletion process (S333). The client device 300 displays the updated two-dimensional image 314 on the display device 500 (S334).

[0080] Furthermore, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S134). The server device 100 connects the combined data 118 after the unnecessary object deletion process and the newly acquired scan data 112 based on the overlapping portion of the two data, thereby generating new combined data 118 (S135).

[0081] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S335). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S336).

[0082] Next, a positional deviation correction process, which is one of the change processes, will be described. Fig. 10 is a sequence diagram for explaining the positional deviation correction process in the three-dimensional scanner system 1 according to the embodiment. Fig. 11 is a schematic diagram for explaining an example of the positional deviation correction process in the three-dimensional scanner system 1 according to the embodiment.

[0083] The server device 100 converts the newly transmitted scan data 112 from the scanner 200 into point cloud data 114 (point cloud data conversion, S140). The server device 100 generates new combined data 118 by splicing together the combined data 118 and the newly acquired scan data 112 based on the overlapping portions between the two data (S141). Specifically, the server device 100 generates combined data G by splicing together the scan data and the combined data based on the overlapping portions between the combined data and the scan data, as shown in FIG.

[0084] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S340). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S341).

[0085] However, the two-dimensional image 314 displayed on the display device 500 includes a portion where a positional deviation of the three-dimensional shape has occurred in the combined data 118. Specifically, as shown in FIG. 11 , there is a portion where a positional deviation of the three-dimensional shape has occurred in the combined data H. Therefore, if the surgeon wants to correct the positional deviation of the three-dimensional shape, the surgeon can set up automatic correction of the positional deviation of the three-dimensional shape from the client device 300. The client device 300 receives setting information for automatically correcting the positional deviation of the three-dimensional shape selected by the surgeon using the mouse 652 (S342). Note that the setting for automatically correcting the positional deviation of the three-dimensional shape can also be selected before starting scanning. Furthermore, the client device 300 transmits setting information for automatically correcting the positional deviation of the three-dimensional shape to the server device 100.

[0086] The server device 100 detects the location of the misalignment of the three-dimensional shape in the combined data 118 based on the setting information for automatically correcting the misalignment of the three-dimensional shape transmitted from the client device 300 (S142). A known algorithm can be used as the algorithm for correcting the misalignment of the three-dimensional shape. The server device 100 then changes the detected location of the misalignment of the three-dimensional shape to corrected combined data 118 (S143). Specifically, as shown in FIG. 11 , the server device 100 changes combined data H to combined data Ha, which corrects the location of the misalignment of the three-dimensional shape. Furthermore, the server device 100 generates corrected coordinate transformation data from the previous and next combined data 118 that have been changed by the misalignment correction process, and transmits the corrected coordinate transformation data to the client device 300. The server device 100 may generate difference data 119 representing the difference between the previous and next combined data 118 and transmit the difference data 119 to the client device 300; however, the amount of data transmitted will be larger than if corrected coordinate transformation data were transmitted. Here, the corrected coordinate transformation data is, for example, the correspondence data between the coordinates before and after the correction, or a coordinate transformation function.

[0087] The client device 300 updates the coordinates of each point of the pre-correction two-dimensional image 314 to the coordinates of each point of the corrected two-dimensional image 314 based on the corrected coordinate transformation data generated by the positional deviation correction process (S343). The client device 300 displays the updated two-dimensional image 314 on the display device 500 (S344).

[0088] Furthermore, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S144). The server device 100 connects the combined data 118 after the misalignment correction process and the newly acquired scan data 112 based on the overlapping portion of the two data, thereby generating new combined data 118 (S145).

[0089] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S345). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S346).

[0090] Next, margin line processing, which is one of the change processes, will be described. Fig. 12 is a sequence diagram for explaining margin line processing in the three-dimensional scanner system 1 according to the embodiment. Here, margin line processing is processing for clarifying the edges of restorations and prostheses in three-dimensional data of the dental arch.

[0091] The server device 100 converts the newly transmitted scan data 112 from the scanner 200 into point cloud data 114 (point cloud data conversion, S150). The server device 100 connects the combined data 118 and the newly acquired scan data 112 based on the overlapping portion of the two data to generate new combined data 118 (S151).

[0092] When generating the combined data 118, the server device 100 generates difference data 119 between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S350). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S351).

[0093] However, in the combined dental arch data 118 generated from the acquired scan data 112, the margins of the restoration / prosthesis may be unclear. Therefore, the surgeon can determine the margin line in the combined dental arch data 118 while viewing the two-dimensional image 314 displayed on the display device 500. The client device 300 receives information indicating that the surgeon has selected one or more points in the combined dental arch data 118 that are thought to be the margins of the restoration / prosthesis using the mouse 652 (margin line setting: S352). Furthermore, the client device 300 transmits margin line setting information including information on the one or more points selected by the surgeon to the server device 100.

[0094] The server device 100 calculates a margin line in the combined data 118 based on the margin line setting information transmitted from the client device 300 (S152). Note that a known algorithm can be used as the algorithm for calculating the margin line. The server device 100 modifies the combined data 118 to include the calculated margin line (S153). Furthermore, the server device 100 generates differential data 119, which is the difference between the combined data 118 before and after the modification by the margin line processing, and transmits the differential data 119 to the client device 300. Note that in step S153, the server device 100 may transmit only the margin line data to the client device 300 without modifying the combined data 118 to include the calculated margin line.

[0095] The client device 300 updates the two-dimensional image 314 based on the difference data 119 generated by the margin line processing (S353). The client device 300 displays the updated two-dimensional image 314 on the display device 500 (S354).

[0096] Furthermore, the server device 100 converts the scan data 112 newly transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S154). The server device 100 connects the combined data 118 after margin line processing and the newly acquired scan data 112 based on the overlapping portion of the two data to generate new combined data 118 (S155).

[0097] When generating the combined data 118, the server device 100 generates difference data 119, which is the difference between the combined data 118 before and after splicing the scan data 112, and transmits the difference data 119 to the client device 300. The client device 300 accumulates the difference data 119 transmitted from the server device 100 and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated difference data 119 (S355). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S356). Note that in the margin line processing shown in FIG. 12, the margin line is calculated using a known algorithm before the creation of the mesh data for storage shown in FIG. 4, but the present invention is not limited to this, and the margin line may be calculated after the mesh data for storage is created.

[0098] More specifically, when scanning a patient's dental arch using the scanner 200 in the three-dimensional scanner system 1, the scanning process is performed in a workflow including upper jaw (or lower jaw) scan, lower jaw (or upper jaw) scan, bite scan, post-processing (creation of mesh data to be saved), analysis, margin line processing, and saving. Fig. 13 is a sequence diagram for explaining an example of dental arch scanning process in the three-dimensional scanner system 1 according to the embodiment. First, upon receiving a scan start operation from the operator, the scanner 200 starts acquiring scan data 112 (three-dimensional data) of the target object in the oral cavity (S201).

[0099] The scanner 200 scans the upper (or lower) dental arch in the oral cavity to sequentially obtain scan data 112 of the upper (or lower) dental arch, and sequentially transmits the obtained scan data 112 to the server device 100. The server device 100 converts the scan data 112 sequentially transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S160). The server device 100 first generates display mesh data 116 corresponding to the point cloud data 114, and then generates combined data 118 by stitching together the two data based on the overlapping portions with the newly acquired scan data 112 (S161). The server device 100 transmits the display mesh data 116 to the client device 300, and then transmits to the client device 300 difference data 119 between the combined data 118 before and after stitching together the scan data 112.

[0100] The client device 300 generates a two-dimensional image 314 of the scanned upper (or lower) dental arch viewed from an arbitrary viewpoint based on the received display mesh data 116 or difference data 119 (S360). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S361).

[0101] Next, the scanner 200 scans the lower (or upper) dental arch in the oral cavity to sequentially obtain scan data 112 of the lower (or upper) dental arch, and sequentially transmits the obtained scan data 112 to the server device 100. The server device 100 converts the scan data 112 sequentially transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S162). The server device 100 first generates display mesh data 116 corresponding to the point cloud data 114, and then generates combined data 118 by stitching together the two data based on the overlapping portions with the newly acquired scan data 112 (S163). The server device 100 transmits the display mesh data 116 to the client device 300, and then transmits to the client device 300 difference data 119 between the combined data 118 before and after stitching together the scan data 112.

[0102] The client device 300 generates a two-dimensional image 314 of the scanned dental arch of the mandible (or maxilla) viewed from an arbitrary viewpoint based on the received display mesh data 116 or difference data 119 (S362). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S363).

[0103] Next, the scanner 200 scans the bite of the dental arch in the oral cavity to sequentially obtain scan data 112 of the bite of the dental arch, and sequentially transmits the obtained scan data 112 to the server device 100. The server device 100 converts the scan data 112 sequentially transmitted from the scanner 200 into point cloud data 114 (point cloud data conversion, S164). The server device 100 first generates display mesh data 116 corresponding to the point cloud data 114, and then generates combined data 118 by joining the two data based on the overlapping portions with the newly acquired scan data 112 (S165). The server device 100 transmits the display mesh data 116 to the client device 300, and then transmits to the client device 300 difference data 119 between the combined data 118 before and after joining the scan data 112.

[0104] The client device 300 generates a two-dimensional image 314 of the scanned dental arch bite viewed from an arbitrary viewpoint based on the received display mesh data 116 or difference data 119 (S364). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S365).

[0105] Next, upon receiving an operation from the operator to end the scan, the scanner 200 ends acquisition of the scan data 112 (three-dimensional data) of the intraoral object (S202). After the scan ends, the server device 100 generates mesh data for storage 120 using all of the scan data 112 acquired by the scanner 200 (S166). The server device 100 stores the generated mesh data for storage 120 in the storage 110 (S167).

[0106] When the server device 100 generates the storage mesh data 120, it transmits the storage mesh data 120 to the client device 300. The client device 300 accumulates the storage mesh data 120 transmitted from the server device 100, and generates a two-dimensional image 314 of the dental arch viewed from an arbitrary viewpoint based on the accumulated storage mesh data 120 (S366). The client device 300 displays the generated two-dimensional image 314 on the display device 500 (S367).

[0107] Next, the server device 100 may perform an analysis process desired by the user, although this is not shown in Fig. 13. Furthermore, the server device 100 calculates a margin line in the combined data 118 based on the margin line setting information transmitted from the client device 300 (S168). The server device 100 transmits data of the calculated margin line to the client device 300.

[0108] The client device 300 updates the two-dimensional image 314 of the dental arch including the margin line based on the margin line data calculated by the margin line processing (S368). The client device 300 displays the updated two-dimensional image 314 on the display device 500 (S369).

[0109] When the dental arch scanning process is completed, the server device 100 stores the analyzed data, the calculated margin line data, and the like in addition to the mesh data for storage 120 in the storage 110 (S169).

[0110] In the three-dimensional scanner system 1, when processing such as trimming is performed, it is performed after each scan, such as upper jaw (or lower jaw) scan, trimming, lower jaw (or upper jaw) scan, trimming, and not after post-processing (creation of mesh data to be saved). However, this is not limited to this, and the three-dimensional scanner system 1 may perform trimming after all scans are completed and before post-processing (creation of mesh data to be saved), or after post-processing (creation of mesh data to be saved).

[0111] Additionally, in the three-dimensional scanner system 1, the margin line processing has been described as being performed after post-processing (creation of mesh data to be saved), but it may also be performed before post-processing (creation of mesh data to be saved). In the three-dimensional scanner system 1, it is sufficient that various processes are performed between the server device 100 and the client device 300, regardless of whether mesh data to be saved is created or not.

[0112] [Variations] The configuration of the three-dimensional scanner system is not limited to the configuration shown in Fig. 1, and various other configurations are possible. Below, modified configurations of the three-dimensional scanner system will be described with reference to the drawings. Fig. 14 is a schematic diagram showing the configuration of a three-dimensional scanner system 1A according to a modified example. In the three-dimensional scanner system 1A, three chair units 400a to 400c are provided in the hospital, and when one scanner 200 is to be used, a LAN cable 220 is connected to the sockets 601a to 601c provided in each of the chair units 400a to 400c.

[0113] The sockets 601a to 601c are network-connected to the server device 100 via a network hub 610. If the network hub 610 has a PoE function, it can supply power to the scanner 200 through a LAN cable 220, but if the network hub 610 does not have the PoE function, an AC adapter 211 is connected to the cradle 210 as shown in FIG.

[0114] In the three-dimensional scanner system 1A, the client devices 300a-300b provided in each chair unit 400a-400b are network-connected to the server device 100 via a network hub 620. A management server device 100a and a wireless communication access device 630 are further connected to the network hub 620. Before the three-dimensional scanner system 1A was introduced into the hospital, the management server device 100a and client devices 300a-300b had already been introduced. The management server device 100a manages patient information, image data, examination data, electronic medical records, etc., and can display patient information, etc. on the display devices 500a-500b provided in each chair unit 400a-400b via the client devices 300a-300b as needed.

[0115] In the three-dimensional scanner system 1A, the three-dimensional shape data processed by the server device 100 can be displayed on the display devices 500a to 500b using the client devices 300a to 300b that have already been installed. Furthermore, by installing a web application program that runs on a web browser on the display devices 500a to 500b, the client devices 300a to 300b that have already been installed can be used as is without installing a dedicated application program, thereby reducing the introduction cost of the three-dimensional scanner system 1A.

[0116] Furthermore, it is not necessary to provide a stationary client device like the chair units 400a to 400b, and a tablet terminal 300c or a notebook PC 300d may be provided in the chair unit 400c as shown in Fig. 14. The tablet terminal 300c and the notebook PC 300d are not connected to the network hub 620 by wire, but are connected to the access device 630 by wireless communication.

[0117] In the 3D scanner system 1A, for example, by connecting to a workstation of a general-purpose image diagnostic device via the network hub 620, it is possible to link and manage 3D data of the dental arch with X-ray images, CT (Computed Tomography) images, OCT (Optical Coherence Tomography) images, etc. Furthermore, in the 3D scanner system 1A, if the management server device 100a functions as a prescription computer or electronic medical record system, it is possible to link and manage 3D data of the dental arch with patient information, test data, medical fee statements, electronic medical records, etc. Furthermore, the 3D scanner system 1A may also manage data obtained from other devices such as an intraoral camera or face scan.

[0118] In the three-dimensional scanner system 1A, the server device 100 and the scanner 200 are connected by wire, but they may also be connected by wireless communication. In the three-dimensional scanner system 1A, only one scanner 200 is installed in the hospital, so the server device 100 and the scanner 200 are connected one-to-one. However, the three-dimensional scanner system 1A may also have multiple scanners 200 installed in the hospital, and the server device 100 and the multiple scanners 200 may be connected one-to-many.

[0119] The three-dimensional scanner system 1A may be capable of displaying a cross-sectional view of the three-dimensional shape of the scanned dental arch on the display device 500. The three-dimensional scanner system 1A may also be capable of measuring the clearance between the upper and lower jaws in the occlusal state of the scanned dental arch. The three-dimensional scanner system 1A may also be capable of displaying undercut portions in a specific direction of the scanned dental arch on the display device 500. The three-dimensional scanner system 1A may also be capable of measuring any distance or angle of the three-dimensional shape of the scanned dental arch.

[0120] The three-dimensional scanner system 1A may be linked to dental laboratory instruction creation software. Also, the three-dimensional scanner system 1A may be linked to software or services that transmit and receive data and messages, such as three-dimensional data and dental laboratory instructions, between dental laboratories.

[0121] The display device 500 is a display provided on the chair units 400a to 400b, but may also be a display provided separately from the chair units 400a to 400b. The three-dimensional scanner system 1A may use a display of a notebook PC, a display of a tablet terminal, or a head-mounted display instead of the display device 500.

[0122] In the three-dimensional scanner system 1, 1A, data such as the scan data 112, the combined data 118, and the storage mesh data 120 may be stored in a large-capacity storage device installed in the hospital or in a cloud storage device.

[0123] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Note that the configurations exemplified in the embodiments and the configurations exemplified in the modified examples can be combined as appropriate. [Explanation of symbols]

[0124] 1,1A Three-dimensional scanner system, 100 Server device, 100a Management server device, 102,305 LAN interface, 103,301 Display interface, 105,303 Peripheral device interface, 106,306 Wireless communication interface, 107,307 Media reader, 109,309 Memory, 110,310 Storage, 112 Scan data, 114 Point cloud data, 116 Display mesh data, 118 Combined data, 119 Differential data, 120 Saved mesh data, 122 Deleted data, 124 Scan program, 126 Web server program, 130,330 Arithmetic circuit, 140,500,500a,500b Display device, 150,350 Removable disk, 160,651 Keyboard, 161,652 Mouse, 200 Scanner, 210 Cradle, 211 AC adapter, 220 LAN cable, 230 wagon, 300, 300a, 300b client device, 300c tablet terminal, 300d notebook PC, 314 two-dimensional image, 321 web application program, 400, 400a, 400b, 400c chair unit, 601, 601a, 601c outlet, 630 access device, 600, 610, 620 network hub.

Claims

1. An information processing system for processing three-dimensional data, a scanner that scans an object to sequentially acquire three-dimensional data of the object; a server device that, when connecting three-dimensional data based on overlapping portions of the three-dimensional data sequentially transmitted from the scanner to generate connected data indicating the three-dimensional shape of the object, generates difference data that is a difference between the connected data before and after connecting the three-dimensional data; an information processing system comprising: a client device that accumulates the difference data sequentially transmitted from the server device, and generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data.

2. The information processing system according to claim 1 , wherein the scanner acquires three-dimensional data of the object in the oral cavity, and the object is a dental arch or a dental arch model in the oral cavity.

3. The information processing system according to claim 1 , wherein the three-dimensional data is point cloud data composed of points indicating three-dimensional coordinate positions.

4. The information processing system according to claim 1 , wherein the server device processes the combined data into mesh data.

5. the client device includes an input unit that accepts user operations; The information processing system according to claim 1 , wherein the server device performs a change process on the combined data based on a user operation received by the input unit.

6. the server device generates, as the difference data, a difference between the combined data before and after the change by the change processing; The information processing system according to claim 5 , wherein the client device updates the two-dimensional image based on the difference data generated by the change process.

7. 7. The information processing system according to claim 6, wherein the change process, when a user uses the input unit to receive information specifying a range to be deleted within the combined data, causes the server device to change the data in the range to be deleted to the combined data from which the data has been deleted.

8. 7. The information processing system of claim 6, wherein the change process, when receiving information on a setting to automatically remove unnecessary objects at the input unit, identifies pre-set deletion data for unnecessary objects from the combined data, and changes the identified deletion data to the deleted combined data.

9. 7. The information processing system of claim 6, wherein, when the input unit receives setting information for automatically correcting a positional misalignment of a three-dimensional shape within the combined data, the server device detects the positional misalignment and changes the combined data to one in which the detected positional misalignment has been corrected.

10. 7. The information processing system of claim 6, wherein the modification process, when received at the input unit from a user selecting one or more points within the combined data, causes the server device to calculate a margin line including the selected points and modify the combined data to include the margin line, or generates data for the margin line without modifying the combined data.

11. the server device has a function as a Web server, and when the combined data is generated by the Web server, the server device generates the difference data before and after joining the three-dimensional data; 11. The information processing system according to claim 1, wherein the client device generates the two-dimensional image based on the difference data through processing of a web application.

12. The server device An information processing system according to any one of claims 1 to 10, wherein, during scanning of the object by the scanner, some of the three-dimensional data transmitted in sequence from the scanner are joined together to generate the combined data, and the difference data before and after joining the three-dimensional data is generated.

13. The information processing system according to claim 12 , wherein the server device updates the combined data by connecting all of the three-dimensional data transmitted in order from the scanner after the scanner has finished scanning the object.

14. A server device that transmits and receives data to and from a client device included in an information processing system that processes three-dimensional data, a receiving unit that receives three-dimensional data sequentially transmitted from a scanner that scans an object and sequentially acquires three-dimensional data of the object; a calculation unit that, when connecting three-dimensional data based on overlapping portions of the three-dimensional data sequentially received by the receiving unit to generate connected data indicating the three-dimensional shape of the object, generates difference data that is a difference between the connected data before and after connecting the three-dimensional data; a transmitting unit that accumulates the difference data generated by the calculation unit and transmits the difference data to a client device that generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data.

15. A client device that transmits and receives data to and from a server device included in an information processing system that processes three-dimensional data, a receiving unit that receives differential data transmitted from the server device, the receiving unit generating differential data representing the difference between the combined data before and after the three-dimensional data is joined together, when the combined data representing the three-dimensional shape of the object is generated by joining three-dimensional data based on overlapping portions of the three-dimensional data sequentially transmitted from a scanner that scans the object and sequentially acquires three-dimensional data of the object; a calculation unit that accumulates the difference data received by the receiving unit and generates a two-dimensional image of the object viewed from an arbitrary viewpoint based on the accumulated difference data; a display unit that displays the two-dimensional image.

Images