Three-dimensional head model generation system, program and three-dimensional head model generation method

Abstract

To provide a three-dimensional head model generation system, program and three-dimensional head model generation method capable of generating an accurate three-dimensional model even when generating a three-dimensional model of a head of an object person who is difficult to keep a stationary state.SOLUTION: A three-dimensional head model generation system 1 comprises: an electronic terminal 2 including an imaging unit 21 for capturing an image of a head 41 and a processing unit 20 for processing the captured image of the head 41; and a server 3, where the processing unit 20 of the electronic terminal 2 includes a terminal-side image acquisition unit 201 that acquires a plurality of images of the head 41 captured from different directions by the imaging unit 21, and a filtering processing unit 205 that determines whether or not the images acquired by the terminal-side image acquisition unit 201 are suitable images satisfying prescribed conditions, and where the server 3 includes a three-dimensional model generation unit 303 that generates a three-dimensional model representing a three-dimensional external shape of the head 41 on the basis of the plurality of suitable images.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a system, a program, and a method for generating a three-dimensional head model. [Background technology]

[0002] Conventionally, cranial shape correction helmets that correct the shape of the head of subjects such as infants who have cranial deformities that require treatment, such as plagiocephaly, have been known. Patent Document 1, for example, describes this type of cranial shape correction helmet.

[0003] Patent document 1 describes a skull correction helmet for babies that is designed based on the external shape of the baby's head and includes an exterior member for pressing against the head to correct the skull shape, and an interior member attached to the inside of the exterior member for cushioning the irritation to the head caused by the exterior member. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-183330 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, cranial shape correction helmets are manufactured based on a 3D model that represents the three-dimensional external shape of the head, which is generated from multiple images of the subject's head taken from different angles. In this case, it is important to accurately read information about the external shape of the head from the images in order to generate an accurate 3D model. However, when the subject is an infant or young child, the infant may move during image capture, making it difficult to accurately read information about the external shape of the head from the captured image.

[0006] The present invention aims to provide a system, program, and method for generating a three-dimensional head model that can generate an accurate three-dimensional model even when generating a three-dimensional model of the head of a subject who is difficult to keep still. [Means for solving the problem]

[0007] The head 3D model generation system comprises an electronic terminal having a photographing unit that photographs the subject's head and a processing unit that processes the images photographed by the photographing unit, and a server that can communicate with the electronic terminal, wherein the processing unit of the electronic terminal has an image acquisition unit that acquires multiple images of the head photographed from different directions by the photographing unit, and a filtering processing unit that determines whether the images acquired by the image acquisition unit are suitable images that satisfy predetermined conditions, and the server has a 3D model generation unit that generates a 3D model that shows the three-dimensional external shape of the head based on the multiple suitable images that are determined by the filtering processing unit to satisfy the predetermined conditions. [Effects of the Invention]

[0008] According to the present invention, even when generating a three-dimensional model of the head of a subject who is difficult to keep still, an accurate three-dimensional model can be generated. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing a system for generating a three-dimensional head model according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of a server in the head 3D model generation system according to one embodiment of the present invention. [Figure 3] 1 is a block diagram showing the hardware configuration of an electronic terminal in a three-dimensional head model generation system according to an embodiment of the present invention. [Figure 4] FIG. 10 is a schematic diagram showing the display unit of the electronic terminal when photographing the head. [Figure 5] 5 is a schematic diagram showing an example of an image captured by a photographing unit of the electronic terminal in the state shown in FIG. 4. FIG. [Figure 6] 1 is a block diagram showing the functional block configuration of an electronic terminal in a three-dimensional head model generation system according to an embodiment of the present invention. [Figure 7A] FIG. 10 is a diagram showing an image in which the subject is stationary and in focus. [Figure 7B] FIG. 7B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 7A. [Figure 8A] FIG. 10 is a diagram showing an image in which the subject is stationary but out of focus. [Figure 8B] FIG. 8B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 8A. [Figure 9A] FIG. 10 is a diagram showing an image captured while the subject is moving. [Figure 9B] FIG. 9B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 9A. [Figure 10] FIG. 10 is an explanatory diagram of conditions regarding the distance and angle between the head and the imaging unit. [Figure 11] FIG. 10 is an explanatory diagram of conditions regarding the distance between the head and the imaging unit. [Figure 12] 1 is a block diagram showing the functional block configuration of a server in a three-dimensional head model generation system according to an embodiment of the present invention. [Figure 13] 10 is a flowchart showing an example of a three-dimensional model generation process executed by an electronic terminal according to an embodiment of the present invention. [Figure 14] 10 is a flowchart illustrating an example of a three-dimensional model generation process executed by a server according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings. Fig. 1 is a schematic diagram showing a head 3D model generation system 1 and an infant 4 according to the present embodiment.

[0011] The head 3D model generation system 1 according to this embodiment is a system for generating a 3D model showing the three-dimensional external shape of the head 41 of a subject, such as an infant or toddler 4. In this embodiment, the generation of a 3D model used when creating a cranial shape correction helmet will be described as an example.

[0012] The cranial shape correction helmet will now be described. A cranial shape correction helmet is a device that, when worn on the head 41 of an infant or toddler 4 as a target subject, promotes deformation so that the shape of the skull is corrected as the skull grows. Target infants or toddlers 4 include, for example, infants and toddlers with cranial deformities requiring treatment, such as plagiocephaly, brachycephaly, or dolichocephaly. Plagiocephaly is a deformed shape in which the skull is not symmetrical but tilted to one side. Brachycephaly is a deformed shape in which the skull is significantly short in the anterior-posterior direction. Dolichocephaly is a deformed shape in which the skull is significantly long in the anterior-posterior direction. The target subject is not particularly limited, and may be a person other than an infant or toddler, a child, or an adult.

[0013] The cranial shape correction helmet is designed based on a three-dimensional model of the ideal head 41 (hereinafter referred to as the ideal three-dimensional model) that is the treatment target for each subject. The ideal three-dimensional model of the head 41 is created based on a three-dimensional model of the actual head 41 (hereinafter simply referred to as the three-dimensional model) that is generated using data on the external shape of the head 41 of each individual infant 4, such as an image of the head 41. For this reason, when creating the cranial shape correction helmet, it is important to obtain accurate data on the external shape of the head 41 of the infant 4 and generate an accurate three-dimensional model.

[0014] The following describes the three-dimensional head model generation system 1. The three-dimensional head model generation system 1 includes an electronic terminal 2 and a server 3 that can communicate with the electronic terminal 2, as shown in FIG.

[0015] The electronic terminal 2 is a device that acquires multiple images of the head 41 taken from different directions and performs preprocessing to generate a 3D model. The electronic terminal 2 is not particularly limited as long as it has a function for taking images and a communication function. Examples of the electronic terminal 2 include a smartphone and a tablet.

[0016] The server 3 is a device that generates a three-dimensional model based on a plurality of images received from the electronic terminal 2.

[0017] An example of the hardware configuration of the server 3 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the hardware configuration of the server 3 in the head 3D model generation system 1.

[0018] 2, the server 3 includes a computer 33, a storage unit 31, and a communication unit 32. A bus 334 and the like connect these units together.

[0019] The computer 33 includes a processor 331 and a read-only memory (ROM) 332 and a random-access memory (RAM) 333 as main storage devices. The processor 331 may be a central processing unit (CPU), a microprocessing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 331 may be a combination of these. The processor 331 may also be a combination of these with a hardware accelerator or the like. The processor 331 controls each unit to realize various functions of the server 3 based on programs such as firmware, system software, and application software stored in the ROM 332, the RAM 333, or an auxiliary storage device that is part of the storage unit 31. Note that some or all of the programs may be incorporated into the circuitry of the processor 331.

[0020] The storage unit 31 is a storage area for various programs and various data for causing the hardware group to function as the server 3, and can be configured with a ROM, RAM, flash memory, a solid-state drive (SSD), a hard disk drive (HDD), etc. Specifically, the storage unit 31 stores programs for causing the computer 33 to execute each function of this embodiment, images acquired from the electronic terminal 2, a generated 3D model of the head 41, etc.

[0021] The communication unit 32 executes processing for the server 3 to communicate with the electronic terminal 2 and other devices via a network. The communication unit 32 of this embodiment is configured to be capable of wireless communication between the electronic terminal 2 and the server 3.

[0022] An example of the hardware configuration of the electronic terminal 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the hardware configuration of the electronic terminal 2 in the head 3D model generation system 1.

[0023] 3, the electronic terminal 2 includes a computer 26, an image capturing unit 21, a storage unit 22, a communication unit 23, an input unit 24, and a display unit 25. A bus 264 and the like connect these units together.

[0024] The computer 26 includes a processor 261 and a read-only memory (ROM) 262 and a random-access memory (RAM) 263 as main storage devices. The processor 261 may be a central processing unit (CPU), a microprocessing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 261 may be a combination of these. The processor 261 may also be a combination of these with a hardware accelerator or the like. The processor 261 controls each unit to realize various functions of the electronic terminal 2 based on programs such as firmware, system software, and application software stored in the ROM 262, the RAM 263, or an auxiliary storage device that is part of the storage unit 22. Note that some or all of the programs may be incorporated into the processor's circuitry.

[0025] The photographing unit 21 is a device that photographs a subject, and mainly includes a lens 211, an adjustment mechanism 212, an image sensor 213, and an A / D conversion unit 214.

[0026] The lens 211 is configured to focus light in order to capture an image of a subject. Examples of the lens 211 include a lens that forms an image of a subject on the light receiving surface of the image sensor 213 and a lens that can freely change the focal length within a certain range.

[0027] The adjustment mechanism 212 is a mechanism for adjusting the exposure time, aperture, ISO sensitivity, focal length, and the like.

[0028] The image sensor 213 forms an image of a subject on a light receiving surface where pixels are arranged two-dimensionally in a matrix, and converts the image into an electrical signal. The image sensor 213 may be, for example, a CMOS image sensor or a CCD image sensor.

[0029] The A / D conversion unit 214 converts the electrical signal read out from the image sensor 213 into a digital signal. An image is formed from the digital signal converted by the A / D conversion unit 214. The image captured by the imaging unit 21 is transmitted to the computer 26, the storage unit 22, the display unit 25, etc.

[0030] The storage unit 22 is a storage area for various programs and various data for causing the hardware group to function as the electronic terminal 2, and can be configured with a ROM, RAM, flash memory, a solid-state drive (SSD), a hard disk drive (HDD), etc. Specifically, the storage unit 22 stores programs for causing the computer 26 to execute each function of this embodiment, the size of the image sensor 213 of the imaging unit 21, the size and shape of an identification marker 52 (described later), quality standards for images (described later), etc.

[0031] The communication unit 23 executes processing for the electronic terminal 2 to communicate with the server 3 and other devices. The communication unit 23 of this embodiment is configured to be able to perform wireless communication between the electronic terminal 2 and the server 3.

[0032] The input unit 24 is a user interface electrically connected to the computer 26. The input unit 24 is composed of buttons and a display 27. The display 27 is composed of, for example, a liquid crystal display (LCD) or an organic electro-luminescent (EL) display, and a touch panel that detects the position touched by the user is provided on the image display surface of the display 27. The user can input information by touching the image display surface of the display 27.

[0033] The display unit 25 is a user interface electrically connected to the computer 26. The display unit 25 is configured with a display 27. The display unit 25 displays images and various information transmitted from the photographing unit 21, the storage unit 22, the computer 26, etc.

[0034] The imaging unit 21 of this embodiment may have a function of capturing a subject such as the head 41 as a still image, a function of capturing a video, or a function of capturing both a still image and a video. Note that in this specification, an image means a still image.

[0035] Here, how a user captures an image of the head 41 of the infant 4 using the electronic terminal 2 will be described with reference to Figures 4 and 5. Figure 4 is a schematic diagram showing an example of the display screen of the display unit 25 when the electronic terminal 2 captures an image of the head 41 of the infant 4. Figure 5 is a schematic diagram showing an example of an image captured by the imaging unit 21 in the state shown in Figure 4. Note that users of the electronic terminal 2 include, for example, doctors and guardians of the infant 4.

[0036] When photographing the head 41, the shape including the hair etc. is acquired as the external shape of the head 41, so a cap 5 is attached to the head 41 to hold the hair down against the scalp and reduce the influence of the hair on the image of the head 41.

[0037] The cap 5 will now be described. The cap 5 is made of a stretchable sheet 50 that can cover the head 41. In this embodiment, an image of the head 41 covered with the sheet 50 is captured by the imaging unit 21.

[0038] The material of the sheet 50 may be, for example, recycled fiber, semi-synthetic fiber, synthetic fiber such as nylon, polyurethane, polyester, or a mixture of two or more of these fibers. The sheet 50 of this embodiment is a cloth-like member made of any of these materials.

[0039] A surface 51 of the portion of the sheet 50 that covers the head 41 is formed with a plurality of identification markers 52 and a plurality of color patterns 53 that can be identified by the photographing unit 21.

[0040] The color pattern 53 is a combination of colors and patterns, and is formed across the entire surface 51 of the sheet 50. The multiple color patterns 53 formed on the surface 51 of the sheet 50 may have the same combination of color and pattern, or may not have the same combination of color and pattern. From the viewpoint of generating an accurate three-dimensional model, it is preferable that all of the color patterns 53 formed on the surface 51 of the sheet 50 have different combinations of color and pattern. In this embodiment, all of the color patterns 53 formed on the surface 51 of the sheet 50 are different from each other. This makes it easier to accurately distinguish each color pattern.

[0041] The colors of the color pattern 53 and background formed on the cap 5 of this embodiment are mainly white, flesh color, pink, red, light blue, orange, yellow-green, green, etc. In Fig. 4, white is represented by white, flesh color by small and sparse dots, pink by small and dense dots, red by large and dense dots, light blue by solid hatching extending diagonally to the upper left, orange by dashed hatching extending diagonally to the upper right, green by solid hatching extending diagonally to the upper right, and yellow-green by diagonal lattice hatching.

[0042] The multiple identification markers 52 are configured to be distinguishable from one another and are formed at intervals across the entire surface 51 of the portion of the seat 50 that covers the head 41. Examples of the identification markers 52 include QR Code (registered trademark) and ArUco markers. The identification marker 52 of this embodiment is an ArUco marker that has corners and has a unique pattern formed on its surface that can be read by the electronic terminal 2. The identification marker 52 of this embodiment is rectangular in plan view. That is, the identification marker 52 of this embodiment has four corners. The multiple identification markers 52 each have a different pattern formed on their surface in black and white.

[0043] The memory unit 22 stores information regarding the positional relationship between each identification marker 52 and the position where each identification marker 52 is formed on the seat 50. Examples of the positional relationship between each identification marker 52 include information regarding each identification marker 52 and its adjacent identification markers 52. For example, information such as the identification marker 52 with identification number 1 being adjacent to and surrounded by the identification markers 52 with identification numbers 2 to 5. When the infant 4 is wearing the cap 5 and the head 41 is covered by the seat 50, each identification marker 52 indicates the positional information of the head 41.

[0044] In this embodiment, 50 identification markers 52 are formed on the surface 51 of the sheet 50. The number of identification markers 52 formed on the surface 51 of the sheet 50 is not particularly limited, and may be less than 50 or may be 50 or more.

[0045] Next, various types of information that are displayed on the display unit 25 when the user uses the electronic terminal 2 to capture an image of the head 41 of the baby 4 will be described.

[0046] In this embodiment, as shown in Fig. 5, a three-dimensional model of the head 41, which is the subject, is generated using a plurality of images in which the entire head 41 is located within a predetermined range (hereinafter referred to as a first detection area) A from the center C of the image, and each identification marker 52 is located within a predetermined range (hereinafter referred to as a second detection area) B from the center of the image. In Fig. 5, the first detection area A is the area inside the chain line, and the second detection area B is the area inside the dashed line.

[0047] 4, display unit 25 displays a live view image, which is a real-time image captured by image sensor 213 of imaging unit 21, when imaging unit 21 is capturing an image. The live view image displayed on display 27 of display unit 25 displays only the image within first detection area A in the captured image. This allows the user to easily capture an image in which the entire head 41 of infant 4, who is the subject, fits within first detection area A.

[0048] The display unit 25 also displays on the display 27 a storage indicator 270 indicating the storage status of identification marker images (described later) in the memory unit 22. The display unit 25 displays, as the storage indicators 270, a front storage indicator 271 indicating the storage status of identification marker images in the front region of the head 41, a rear storage indicator 272 indicating the storage status of identification marker images in the rear region of the head 41, a right storage indicator 273 indicating the storage status of identification marker images in the right region of the head 41, a left storage indicator 274 indicating the storage status of identification marker images in the left region of the head 41, and an overall storage indicator 275 indicating the storage status of identification marker images in the entire region of the head 41. The overall storage indicator 275 displays a 3D model of the head 41, and indicates the locations of the head 41 corresponding to the captured and stored identification markers. These storage indicators 270 allow the user to easily understand the images that need to be captured to generate a 3D model of the head 41.

[0049] Next, a description will be given of the functional configuration of the electronic terminal 2 for performing the 3D model generation process by the electronic terminal 2 of the head 3D model generation system 1. FIG. 6 is a functional block diagram showing part of the functional configuration of the electronic terminal 2.

[0050] The electronic terminal 2 is mainly configured by a processor 261, and includes a processing unit 20 that executes a 3D model generation process. As shown in FIG. 6, the processing unit 20 includes a terminal-side image acquisition unit 201, a filtering processing unit 205, a display control unit 206, and a transmission processing unit 207.

[0051] The terminal-side image acquisition unit 201 includes an acquisition unit 202 , an image position determination unit 203 , and an identification marker detection unit 204 .

[0052] The acquisition unit 202 executes a process of acquiring an image of the head 41 captured by the imaging unit 21. Along with the image of the head 41, the acquisition unit 202 also acquires various parameters of the imaging unit 21, such as the exposure time, aperture, ISO sensitivity, and focal length, which were set by the user or the adjustment mechanism 212 when the head 41 was captured. Note that the acquisition unit 202 may extract a still image from the moving image of the head 41 captured by the imaging unit 21, and acquire the extracted still image as the image.

[0053] The image position determination unit 203 executes a process of determining whether the entire head 41 of the infant 4 is located within the first detection area A on the image acquired by the acquisition unit 202. The image position determination unit 203 may determine whether the entire head 41 on the image is located within the first detection area A using a learning model constructed by, for example, supervised learning. The supervised learning may use, for example, an image of the head 41 of the infant 4 wearing the cap 5 as input data, an evaluation result of whether the entire head 41 is located within the first detection area A in the image as a label, and many pairs of the input data and the label as training data. If the image position determination unit 203 determines that the entire head 41 on the image is not located within the first detection area A, it discards the image.

[0054] The identification marker detection unit 204 executes a process of determining whether or not at least one identification marker 52 on an image in which it has been determined that the entire head 41 is located within the first detection area A is located entirely within the second detection area B. For example, the identification marker detection unit 204 detects the identification marker 52 and its identification information based on information about the identification marker 52 read by the imaging unit 21. Then, the identification marker detection unit 204 determines whether or not the detected identification marker 52 is located within the second detection area B on the image based on a comparison result between the detected identification marker 52 and information about the second detection area B stored in the storage unit 22. If the identification marker detection unit 204 determines that the detected identification marker 52 is located within the second detection area B on the image, it acquires the image as an image corresponding to the detected identification marker 52 (hereinafter referred to as an identification marker image) and transfers it to the filtering processing unit 205. When multiple identification markers 52 exist within the second detection area B of the image, the identification marker detection unit 204 identifies the identification marker 52 closest to the center C, acquires that image as an identification marker image of the identified identification marker 52, and transfers it to the filtering processing unit 205. On the other hand, when the identification marker detection unit 204 determines that all of the detected identification markers 52 are not located within the second detection area B on the image, it discards that image.

[0055] The filtering processing unit 205 executes filtering processing to determine whether or not the identification marker image acquired by the terminal-side image acquisition unit 201 is a matching image that satisfies predetermined conditions. In this embodiment, the filtering processing unit 205 executes filtering processing when an identification marker image of the same identification marker 52 as the identification marker image transferred from the terminal-side image acquisition unit 201 is not stored in the storage unit 22. On the other hand, when an identification marker image of the same identification marker 52 as the identification marker image transferred from the terminal-side image acquisition unit 201 is stored in the storage unit 22, the filtering processing unit 205 discards the transferred image.

[0056] The filtering process will now be described. The predetermined conditions in the filtering process are image quality standards from the perspective of generating an accurate three-dimensional model. In this embodiment, the image quality standards include three conditions: a condition related to the degree of image blur; a condition related to the distance between the imaging unit 21 and the head 41 when the imaging unit 21 images the head 41; and a condition related to the orientation of the imaging unit 21 with respect to the head 41 when the imaging unit 21 images the head 41. The filtering processing unit 205 in this embodiment determines an identification marker image to be a suitable image if all three of the above image quality standards are satisfied. Note that, in the filtering process, the process of determining the quality standard related to the degree of image blur is referred to as blur determination process, the process of determining the quality standard related to the angle of the imaging unit 21 with respect to the head 41 is referred to as angle determination process, and the process of determining the quality standard related to the distance between the imaging unit 21 and the head 41 is referred to as distance determination process.

[0057] First, the blur determination process will be described with reference to Figures 7A to 9B. If the baby 4 moves the head 41 at the moment an image of the head 41 is captured, the subject, head 41, will often go out of focus, resulting in an out-of-focus image. It becomes difficult to obtain accurate data on the outer shape of the head 41 from such an out-of-focus image. The blur determination process is performed to select only images in which the degree of blur is within an allowable range for generating a three-dimensional model.

[0058] In the blur determination process, the degree of blur in an image is determined based on the average value of the spatial frequencies of the transformed data obtained by Fourier transforming the image. Specifically, in the blur determination process, the following processes (a) to (d) are executed in this order. (a) Transformed data is obtained by Fourier transforming the identification marker image acquired by the terminal-side image acquisition unit 201. The transformed data is obtained, for example, by performing a two-dimensional Fourier transform on the identification marker image, which is a Fourier transform in the horizontal direction and then a Fourier transform in the vertical direction. (b) The converted data acquired in (a) is subjected to a process of removing low spatial frequency components, i.e., spatial frequency components below a predetermined spatial frequency. For example, the process of (b) may be a process of rearranging the converted data acquired in (a) so that spatial frequency components with lower spatial frequencies are located closer to the center of the image, and removing spatial frequency components in a predetermined range area (hereinafter referred to as a masking area) E from the center of the image of the rearranged converted data. (c) Calculate the average value of the spatial frequencies of the transformed data from which the low spatial frequency components have been removed in (b). (d) It is determined whether the average value of the spatial frequency calculated in (c) is equal to or greater than a predetermined threshold value (hereinafter referred to as a blur determination value).

[0059] If the filtering processing unit 205 determines in (d) above that the average value of the spatial frequency is equal to or greater than the blur judgment value, it proceeds to the angle judgment process, and if it determines that the average value of the spatial frequency is less than the blur judgment value, it stops the filtering process and discards the identification marker image that was the subject of the judgment.

[0060] The relationship between spatial frequency and image blur will be described with reference to FIGS. 7A to 9B. FIG. 7A is a diagram showing an image captured when the subject is stationary and in focus. FIG. 8A is a diagram showing an image captured when the subject is stationary but out of focus. FIG. 9A is a diagram showing an image captured when the subject is moving. FIG. 7B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 7A. FIG. 8B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 8A. FIG. 9B is a diagram showing transformed data obtained by Fourier transforming the image shown in FIG. 9A. Specifically, FIGS. 7B, 8B, and 9B are images of transformed data obtained by performing the two-dimensional Fourier transform described above (a) on the images of FIGS. 7A, 8A, and 9A, and rearranging the resulting transformed data so that spatial frequency components with lower spatial frequencies are located closer to the center of the image. Note that in FIGS. 7B, 8B, and 9B, darker white areas indicate lower spatial frequencies, and darker black areas indicate higher spatial frequencies. The area inside the dashed circle shown in FIGS. 7B, 8B, and 9B indicates a masking area E that is removed from the converted data in (b).

[0061] Comparing Figures 7B, 8B, and 9B, there tends to be less difference among the three transformed data in the masking region E, which has a low spatial frequency in the Fourier transformed transformed data. On the other hand, it can be seen that in the transformed data, in the region other than the masking region E, Figure 7B, which shows the transformed data of a stationary, in-focus image, has many regions with high spatial frequency compared to Figures 8B and 9B. Therefore, in the blur determination process, in order to more clearly grasp the difference in spatial frequency between an in-focus image and an out-of-focus image due to, for example, movement of the head 41 during shooting, the data of the masking region E, which has a low spatial frequency, is removed and the spatial frequencies of the transformed data are compared. This allows the degree of blur in the image to be determined more accurately.

[0062] Next, the angle determination process and distance determination process will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is an explanatory diagram of conditions related to the distance and angle between the head 41 and the imaging unit 21. Fig. 11 is an explanatory diagram of conditions related to the distance between the head 41 and the imaging unit 21.

[0063] 10, the angle of the lens 211 of the image capturing unit 21 with respect to the identification marker 52 is significantly different from that of the electronic terminal 2 shown by the dashed line in which the lens 211 of the image capturing unit 21 faces the surface on which the pattern of the identification marker 52 is formed. With an image of the head 41 captured in this state, it is difficult to obtain accurate data on the outer shape of the head 41. Also, if the distance L between the head 41 and the lens 211 of the image capturing unit 21 is too close or too far, it is difficult to obtain accurate data on the outer shape of the head 41 from the image. An angle determination process and a distance determination process are performed to select only images in which the angle and distance of the image capturing unit 21 with respect to the head 41 are within ranges acceptable for generating a 3D model.

[0064] In the angle determination process, the following processes (e) to (h) are executed in this order. (e) In order to grasp the orientation of the identification marker 52, position information u indicating the position of the identification marker 52 on the image is identified from the position information of the corner of the identification marker 52. (f) Using the position information of the identification marker 52 identified in (e), parameter information K including the focal length set in the photographing unit 21 and the size of the image sensor 213, and preset three-dimensional space coordinates X, a perspective transformation matrix Rt shown in the following equation (2) is calculated using the following equation (1). u=K×Rt×X Equation (1)

[0065]

number

[0066] From the perspective transformation matrix Rt calculated in (g) and (f), the orientation of the identification marker 52 is estimated as the orientation of the imaging unit 21 relative to the head 41. Specifically, the rotation matrix R, which is the part related to the rotation of the object, and is shown in the following equation (3) is extracted from the perspective transformation matrix Rt, and the rotation matrix R is converted into Euler angles.

[0067]

number

[0068] If the filtering processing unit 205 determines that all of the Euler angles ψ, θ, and φ obtained in (h) above are within the range of the angle judgment value, it proceeds to distance judgment processing, and if it determines that at least one of the Euler angles ψ, θ, and φ is outside the range of the angle judgment value, it stops the filtering processing and discards the identification marker image that was the subject of judgment.

[0069] In the distance determination process, the above processes (e) and (f) and the following processes (i) to (k) are executed in this order. (i) The components of the translation vector t of the perspective transformation matrix Rt calculated in (f), i.e., the norm of the components shown in the following equation (4), are calculated using the following equation (5). This calculation result indicates the actual distance L between the lens 211 of the imaging unit 21 and the identification marker 52.

[0070]

number

number

[0071] (j) Determine whether the distance L calculated in (i) is within a range of a predetermined threshold (hereinafter referred to as the distance determination value). Note that Fig. 11 shows an example in which the distance L2 is the lower limit value of the distance determination value and the distance L1 is the upper limit value.

[0072] If the filtering processing unit 205 determines that the distance L calculated in (k) above is within the range of the distance judgment value, it determines that the identified marker image being judged is a suitable image that satisfies the quality standard, and stores the identified marker image as a suitable image in the storage unit 22. On the other hand, if it determines that the distance L is outside the range of the distance judgment value, it discards the identified marker image being judged.

[0073] The display control unit 206 executes a process to control the display on the display unit 25 in accordance with the storage status of the matching images in the storage unit 22. For example, when an identification marker image of a new identification marker 52 is stored in the storage unit 22 as a matching image, the display control unit 206 changes the storage display 270 to reflect the storage status of the matching image. Furthermore, when the identification marker images of all predetermined identification markers 52 have been stored in the storage unit 22 as matching images, the display control unit 206 may display a display on the display unit 25 notifying the user of this fact.

[0074] The transmission processing unit 207 executes a process of transmitting a plurality of matching images of the head 41 stored in the storage unit 22 to the server 3. The transmission processing unit 207 transmits the identification marker images of all predetermined identification markers 52 that are stored in the storage unit 22 as matching images to the server 3. For example, when the identification marker images of all identification markers 52 are stored in the storage unit 22, the transmission processing unit 207 may automatically transmit the identification marker images of all of the identification markers 52 to the server 3. For example, when a user performs an operation via the input unit 24 to transmit the matching images in the storage unit 22 to the server 3, the transmission processing unit 207 may transmit images corresponding to all of the identification markers 52 to the server 3 based on the user operation.

[0075] Note that all of the predetermined identification markers 52 may be, for example, all of the identification markers 52 formed over the entire surface 51 of the portion of the seat 50 that covers the head 41.

[0076] Since a series of processes such as photographing, acquiring, filtering, and saving images are performed on a single electronic terminal 2, for example, the time lag between photographing an image and obtaining the result of determining whether the image is suitable or not is reduced, and the large number of images required to create a 3D model can be smoothly acquired.

[0077] Next, a description will be given of the functional configuration of the server 3 for performing the 3D model generation process by the server 3 of the head 3D model generation system 1. FIG.

[0078] The server 3 is mainly configured by a processor 331, and includes a processing unit 30 that executes a 3D model generation process. As shown in Fig. 12, the processing unit 30 includes a server-side image acquisition unit 301, a background processing unit (background removal unit) 302, a 3D model generation unit 303, and a transmission processing unit 304.

[0079] The server-side image acquisition unit 301 executes a process of acquiring matching images of all predetermined identification markers 52 from the electronic terminal 2 via the communication unit 32 .

[0080] The background processing unit 302 performs background removal processing on each of the multiple matching images acquired by the server-side image acquisition unit 301. The background removal processing is a process for removing information from the matching images other than the detection target image, including the head 41 of the infant 4. The detection target image may be, for example, an image including only the head 41 of the infant 4, an image including the portion above the face, an image including the portion above the neck of the infant 4, or an image including the entire infant 4. For example, the background processing unit 302 may perform background removal processing by constructing a learning model for detecting the detection target image in an image using machine learning or the like, detecting the detection target image in the matching image using the constructed learning model, and removing information other than the detected detection target image. For example, the background removal processing may be performed using a program such as "Rembg" or "Detectron 2," which are Python libraries. This removes information from the images other than the detection target image necessary for generating a 3D model, thereby narrowing down the feature amounts used during generation and more accurately detecting overlapping feature points of the head 41 between each image.

[0081] The 3D model generation unit 303 executes a process for generating a 3D model of the head 41 based on multiple matching images. In this embodiment, the 3D model generation unit 303 generates a 3D model of the head 41 using photogrammetry technology based on multiple matching images from which information other than the detection target image has been removed by the background processing unit 302. Specifically, the 3D model generation unit 303 detects overlapping feature points between images corresponding to each identification marker 52 and generates a 3D model by three-dimensionally overlaying the images. At this time, the 3D model generation unit 303 generates a 3D model of the head 41 based on position information on the head 41 indicated by the identification markers 52 and the color and pattern 53 on the surface 51 of the sheet 50. Using the position information indicated by the identification markers 52 makes it easier to determine which part of the head 41 each image represents. Furthermore, because there are no identical color and pattern 53 locations on the head 41, differences in feature amounts become clearer depending on the location, enabling more accurate detection of overlapping feature points between images. Therefore, a more accurate three-dimensional model of the head 41 can be generated.

[0082] The transmission processing unit 304 executes a process of transmitting the 3D model generated by the 3D model generation unit 303 to the electronic terminal 2 or another device. A user of the device such as the electronic terminal 2 that has received the 3D model can confirm the generated 3D model of the head 41 by displaying it on the display 27 or the like.

[0083] Next, an example of the flow of the 3D model generation process executed by the processing unit 20 of the electronic terminal 2 of the head 3D model generation system 1 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the 3D model generation process executed by the electronic terminal 2.

[0084] 13, in step S11, the acquisition unit 202 of the processing unit 20 acquires an image captured by the imaging unit 21. At this time, the acquisition unit 202 also acquires parameter information, including the focal length and the size of the image sensor set in the imaging unit 21 when the image was captured, along with the image.

[0085] In step S12, the image position determination unit 203 determines whether or not the entire head 41 on the image acquired in step S11 is located within the first detection area A. If the image position determination unit 203 determines that the entire head 41 is not located within the first detection area A (NO in step S12), the process returns to step S11. On the other hand, if the image position determination unit 203 determines that the entire head 41 is located within the first detection area A (YES in step S12), the process proceeds to step S13.

[0086] In step S13, the identification marker detection unit 204 detects the identification marker 52 on the image in which it has been determined in step S12 that the entire head 41 is located within the first detection area A. At this time, the identification marker detection unit 204 also acquires the identification information of the detected identification marker 52.

[0087] In step S14, the identification marker detection unit 204 determines whether or not at least one identification marker 52 detected in step S13 is located within the second detection area B. If the identification marker detection unit 204 determines that the detected identification marker 52 is not located within the second detection area B (NO in step S14), the process returns to step S11. On the other hand, if the identification marker detection unit 204 determines that the detected identification marker 52 is located within the second detection area B (YES in step S14), the process proceeds to step S15. At this time, the identification marker detection unit 204 identifies the identification marker 52 that is closest to the center C of the image among the identification markers 52 in the second detection area B.

[0088] In step S15, the filtering processing unit 205 determines whether or not a matching image in which the identification marker 52 identified in step S14 is located at the very center of the image is stored in the storage unit 22. If the filtering processing unit 205 determines that a matching image in which the identification marker 52 is located at the very center of the image is already stored in the storage unit 22 (YES in step S15), the processing returns to step S11. On the other hand, if the filtering processing unit 205 determines that a matching image in which the identification marker 52 is located at the very center of the image is not stored in the storage unit 22 (NO in step S15), the processing proceeds to step S16.

[0089] In step S16, the filtering processing unit 205 executes a blur determination process on the image corresponding to the identification marker 52 identified in step S14. Specifically, the filtering processing unit 205 performs a Fourier transform on the image to obtain transformed data, and calculates the average value of the spatial frequencies of the transformed data by excluding low spatial frequency components from the obtained transformed data.

[0090] In step S17, if the filtering processing unit 205 determines in step S16 that the average value is less than the blur determination value (NO in step S17), the processing returns to step S11. On the other hand, if the filtering processing unit 205 determines in step S16 that the average value is equal to or greater than the blur determination value (YES in step S17), the filtering processing unit 205 determines that the degree of blur in the image is within the allowable range, and proceeds to step S18.

[0091] In step S18, the filtering processing unit 205 executes angle determination processing on the image for which the degree of blur has been determined in step S17 to be within the allowable range (greater than or equal to the blur determination value). Specifically, the filtering processing unit 205 detects the positions of the corners of the identification marker 52, specifies position information of the identification marker 52 on the image based on the detected corner positions, and calculates a perspective transformation matrix using parameter information such as the focal length set in the imaging unit 21 and the size of the image sensor 213, acquired in step S11. Then, a rotation matrix is ​​extracted from the calculated perspective transformation matrix and converted into Euler angles, and it is determined whether or not each of the obtained Euler angles ψ, θ, and φ is within the range of the angle determination value.

[0092] In step S19, if the filtering processing unit 205 determines in step S18 that at least one of the Euler angles ψ, θ, and φ is outside the range of the angle determination value (NO in step S19), the processing returns to step S11. On the other hand, if the filtering processing unit 205 determines in step S18 that all of the Euler angles ψ, θ, and φ are within the range of the angle determination value (YES in step S19), the filtering processing unit 205 determines that the angle of the imaging unit 21 with respect to the identification marker 52 during imaging is within the allowable range, and the processing proceeds to step S20.

[0093] In step S20, the filtering processing unit 205 executes distance determination processing for the image for which the angle was determined to be within the allowable range in step S19. Specifically, the filtering processing unit 205 calculates the actual distance L between the imaging unit 21 and the identification marker 52 at the time of imaging by using the norm of the components of the translation vector from the perspective transformation matrix calculated in step S18 and the above-mentioned equation (5). Then, it is determined whether the calculated distance L is within the range of distance determination values.

[0094] In step S21, if the filtering processing unit 205 determines in step S20 that the distance L is outside the range of the distance determination value (NO in step S21), the processing returns to step S11. On the other hand, if the filtering processing unit 205 determines in step S20 that the distance L is within the range of the distance determination value (YES in step S21), the filtering processing unit 205 determines that the distance L between the identification marker 52 and the imaging unit 21 at the time of imaging is within the allowable range, and the processing proceeds to step S22.

[0095] In step S22, the filtering processing unit 205 determines that the image for which the distance L is determined to be within the allowable range in step S21 is a suitable image, and stores the determined image in the storage unit 22.

[0096] In step S23, the display control unit 206 controls the display unit 25 to reflect, in the display showing the storage status of the identification marker images, that the image for the new identification marker 52 has been stored as a matching image.

[0097] In step S24, the transmission processing unit 207 determines whether or not the identification marker images of all the predetermined identification markers 52 are stored as matching images in the storage unit 22. If the transmission processing unit 207 determines that the identification marker images of all the identification markers 52 are not stored as matching images (NO in step S24), the processing returns to step S11. On the other hand, if the transmission processing unit 207 determines that the identification marker images of all the identification markers 52 are stored as matching images (YES in step S24), the processing proceeds to step S25.

[0098] In step S25, the transmission processing unit 207 transmits all of the matching images of the head 41, which is the target for generating a 3D model this time and which are stored in the storage unit 22, to the server 3. Thereafter, the 3D model generation process in the processing unit 20 ends.

[0099] Next, an example of the flow of the 3D model generation process executed by the server 3 of the head 3D model generation system 1 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the 3D model generation process executed by the server 3.

[0100] As shown in FIG. 14, in step S31, the server-side image acquisition unit 301 of the processing unit 30 acquires all the matching images transmitted in step S25.

[0101] In step S32, the background processing unit 302 performs background removal processing on each of all the matching images acquired in step S25. As a result, information other than the detection target image is removed from the matching images.

[0102] In step S33, the three-dimensional model generation unit 303 executes a process of generating a three-dimensional model of the head 41 using photogrammetry technology, based on the matching image that has been subjected to background removal processing in step S32.

[0103] In step S34, the three-dimensional model generation unit 303 stores the three-dimensional model generated in step S33. After that, the processing unit 30 ends the three-dimensional model generation process.

[0104] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments and can be modified as appropriate.

[0105] In the above embodiment, a cap 5 having an identification marker 52 and a color pattern 53 formed on the surface 51 of the sheet 50 was used when photographing the head 41, but a cap 5 having no identification marker 52 or color pattern 53 formed on the surface 51 of the sheet 50 may also be used, a cap 5 having an identification marker 52 but no color pattern 53 may also be used, or a cap 5 having a color pattern 53 but no identification marker 52 may also be used. Furthermore, instead of the identification marker 52, the cap 5 may simply have a marker that has no identification function and is square in plan view.

[0106] In the above embodiment, the filtering processing unit 205 performs blur determination processing, angle determination processing, and distance determination processing as filtering processing. However, it is also possible to perform only one or two of the three processing steps rather than all three processing steps. That is, it is also possible to perform multiple types of filtering processing to extract suitable images and generate a 3D model based on the extracted suitable images. This allows for the generation of a more accurate 3D model of the head 41.

[0107] In the above embodiment, the filtering processing unit 205 performs the blur determination processing, the angle determination processing, and the distance determination processing in this order, but the order in which the processing is performed is not particularly limited. For example, the filtering processing may be performed in the order of the blur determination processing, the distance determination processing, and the angle determination processing, or the filtering processing may be performed in the order of the distance determination processing, the angle determination processing, and the blur determination processing, or the filtering processing may be performed in the order of the angle determination processing, the distance determination processing, and the blur determination processing, or the filtering processing may be performed in the order of the angle determination processing, the blur determination processing, and the distance determination processing.

[0108] In the above embodiment, the server 3 has the background processing unit 302, but the server 3 may not have the background processing unit 302, and background removal processing may not be performed in the 3D model generation processing. Alternatively, the electronic terminal 2 may have the background processing unit 302 instead of the server 3, and the electronic terminal 2 may transmit a matching image that has undergone background removal processing to the server 3.

[0109] In the above embodiment, the electronic terminal 2 has the filtering processing unit 205, but the server 3 may have the filtering processing unit 205.

[0110] In the above embodiment, the terminal-side image acquisition unit 201 had an acquisition unit 202, an image position determination unit 203, and an identification marker detection unit 204, but it may be configured not to include at least one of the image position determination unit 203 and the identification marker detection unit 204.

[0111] Furthermore, for example, the identification marker detection unit 204 may use a learning model constructed by performing supervised learning to determine whether or not at least one identification marker 52 on an image is entirely located within the second detection area B. In this case, the supervised learning may use, for example, an image including, as an object, an identification marker 52 formed on the surface 51 of a sheet 50 covering the head 41 as input data, an evaluation result of whether or not at least one identification marker 52 in the image is entirely located within the second detection area B as a label, and many pairs of the input data and the label as teaching data.

[0112] In the above embodiment, if an identification marker image of the same identification marker 52 as the identification marker image transferred from the terminal-side image acquisition unit 201 is stored in the memory unit 22, the filtering processing unit 205 discards the transferred image. However, the transferred image may not be discarded, but may instead be stored as a backup in a backup memory area of ​​the memory unit 22.

[0113] According to the above-described embodiment and modified examples, the following effects are achieved.

[0114] (1) The head 3D model generation system 1 comprises an electronic terminal 2 having an imaging unit 21 that photographs the head 41 of an infant or young child 4 and a processing unit 20 that processes the image of the head 41 photographed by the imaging unit 21, and a server 3 that can communicate with the electronic terminal 2. The processing unit 20 of the electronic terminal 2 has a terminal-side image acquisition unit 201 that acquires multiple images of the head 41 photographed from different directions by the imaging unit 21, and a filtering processing unit 205 that determines whether the images acquired by the terminal-side image acquisition unit 201 are suitable images that satisfy predetermined conditions. The server 3 has a 3D model generation unit 303 that generates a 3D model showing the three-dimensional external shape of the head 41 based on the multiple suitable images determined by the filtering processing unit 205 to satisfy the predetermined conditions.

[0115] As a result, even when generating a three-dimensional model of the head 41 of an infant 4, which is difficult to keep still, an accurate three-dimensional model of the head can be generated because the three-dimensional model of the head is generated using only images that satisfy predetermined conditions.

[0116] (2) In the head 3D model generation system 1 described in (1), the predetermined conditions include a condition regarding the degree of blurring of the image.

[0117] As a result, even if the baby 4 moves at the moment of photographing and the image of the head 41 is likely to become blurred, the images to be used for generating the 3D model are selected according to the degree of blurring, so a more accurate 3D model of the head 41 can be generated.

[0118] (3) In the head 3D model generation system 1 described in (2), the filtering processing unit 205 performs a Fourier transform on the image acquired by the terminal-side image acquisition unit 201 to acquire transformed data, and determines the degree of blurring of the image based on the average value of the spatial frequencies in the transformed data excluding the low spatial frequency components.

[0119] This allows for a Fourier transform to be performed, and the spatial frequency components of only the portion excluding the low-frequency spatial frequency components in which it is difficult to confirm differences between images in the obtained transformed data, thereby enabling the degree of blur to be specified more accurately.

[0120] (4) In the head 3D model generation system 1 described in (1), the predetermined conditions include a condition regarding the distance L between the imaging unit 21 and the head 41 when the imaging unit 21 images the head 41.

[0121] As a result, even in situations where it is difficult to photograph an infant 4, who has difficulty remaining still during photography, from various directions under fixed conditions, the images used to generate the 3D model are selected taking into consideration the distance between the photography unit 21 and the head 41, so a more accurate 3D model of the head 41 can be generated.

[0122] (5) In the head 3D model generation system 1 described in (4), the photographing unit 21 photographs the head 41 covered by a sheet, and a plurality of markers are formed at intervals on the surface of the portion of the sheet 50 covering the head 41, and the terminal side image acquisition unit 201 acquires an image in which the marker is located within a second detection area B, which is a predetermined range from the center C, as an image corresponding to the marker, and the filtering processing unit 205 determines the distance between the photographing unit 21 and the marker based on position information indicating the position of the marker on the image corresponding to the marker and parameter information including the focal length set in the photographing unit 21 and the size of the image sensor.

[0123] As a result, the distance is calculated based on the marker located on the center C side of the captured image, so the distance between the imaging unit 21 and the head 41 can be determined more accurately.

[0124] (6) In the head 3D model generation system 1 described in (1), the predetermined conditions include a condition regarding the orientation of the imaging unit 21 with respect to the head 41 when the imaging unit 21 images the head 41.

[0125] As a result, even in situations where it is difficult to photograph an infant 4, who has difficulty in remaining still during photography, from various directions under fixed conditions, the images used to generate the 3D model are selected taking into consideration the angle between the photography unit 21 and the head 41, so a more accurate 3D model of the head 41 can be generated.

[0126] (7) In the head 3D model generation system 1 described in (1), the photographing unit 21 photographs the head 41 covered by the sheet 50, and a plurality of markers are formed at intervals on the surface of the portion of the sheet 50 covering the head 41, and the terminal side image acquisition unit 201 acquires an image corresponding to the marker, in which the marker is located within a second detection area B that is a predetermined range from the center, and the filtering processing unit 205 determines the orientation of the photographing unit 21 relative to the head 41 based on position information indicating the position of the marker on the image corresponding to the marker and parameter information including the focal length set in the photographing unit 21 and the size of the image sensor.

[0127] As a result, the angle is estimated from the information of the marker positioned on the center C side of the captured image, so the angle of the imaging unit 21 relative to the head 41 can be specified more accurately.

[0128] (8) In the head 3D model generation system 1 described in any one of (1) to (7), the photographing unit 21 photographs the head 41 while it is covered with a sheet 50, and on the surface of the portion of the sheet 50 covering the head 41, a plurality of identification markers 52 associated with position information on the head 41 are formed at intervals, and a plurality of color patterns 53 are formed, and the terminal side image acquisition unit 201 acquires an image in which the identification marker 52 is located within a second detection area B that is a predetermined range from the center C as an image corresponding to the identification marker 52, and the filtering processing unit 205 determines whether the image corresponding to the identification marker 52 acquired by the terminal side image acquisition unit 201 is a matching image, and the 3D model generation unit 303 generates a 3D model based on the position information and color patterns 53 associated with the identification marker 52 of the matching images of all images corresponding to the predetermined identification markers 52.

[0129] This allows the positional information of the head 41 associated with the identification marker 52 and the information on the color and pattern 53 to be used to generate a 3D model, making it easier to more accurately determine the positional relationship between each of the multiple matching images, and allowing a more accurate 3D model of the head 41 to be generated.

[0130] In the head 3D model generation system 1 described in (9) and (8), all of the color patterns 53 formed on the surface 51 of the sheet 50 are different from one another.

[0131] As a result, there are no parts with the same color and pattern 53 on the surface 51 of the sheet 50 covering the head 41, so the differences in feature amounts depending on the part of the head 41 become clearer, making it easier to more accurately determine which part of the head 41 the acquired image is from, and a more accurate three-dimensional model of the head 41 can be generated.

[0132] (10) In the head 3D model generation system 1 described in any one of (1) to (9), the predetermined conditions include a condition regarding the distance between the imaging unit 21 and the head 41 when the imaging unit 21 images the head 41, and a condition regarding the orientation of the imaging unit 21 relative to the head 41 when the imaging unit 21 images the head 41.

[0133] As a result, even in situations where it is difficult to photograph an infant 4, who has difficulty remaining still during photography, from various directions under fixed conditions, the images used to generate the 3D model are selected taking into consideration the distance and angle between the photography unit 21 and the head 41, so a more accurate 3D model of the head 41 can be generated.

[0134] (11) In the head three-dimensional model generation system 1 described in any one of (1) to (10), the predetermined conditions further include a condition regarding the degree of blurring of the image.

[0135] This allows the images used to generate the 3D model to be selected based on the degree of blur, resulting in a more accurate 3D model of the head.

[0136] (12) In the head 3D model generation system 1 described in any one of (1) to (11), the filtering processing unit 205 determines whether the image acquired by the terminal side image acquisition unit 201 satisfies a condition regarding the degree of blur, and then determines whether the image determined to satisfy the condition regarding the degree of blur satisfies a condition regarding the distance between the imaging unit 21 and the head 41 when the imaging unit 21 photographs the head 41, and a condition regarding the orientation of the imaging unit 21 relative to the head 41 when the imaging unit 21 photographs the head 41.

[0137] As a result, the distance and angle are determined after selecting an image according to the degree of blur, and the distance and angle are determined using a clear image, so that the distance and angle can be specified more accurately. Therefore, the distance and angle can be determined more accurately, and a more accurate 3D model of the head 41 can be generated.

[0138] (13) In the head 3D model generation system 1 described in any one of (1) to (12), the server 3 further has a background processing unit 302 that detects a detection target image including the head 41 of the infant 4 on the matching image and removes information other than the detection target image on the matching image, and the 3D model generation unit 303 generates a 3D model showing the three-dimensional external shape of the head 41 based on the multiple matching images from which information other than the detection target image has been removed by the background processing unit 302.

[0139] This allows the use of a compatible image in which only information from the detection target image including the head 41 of the infant 4 is extracted, thereby narrowing down the features required to generate a 3D model of the head 41 and generating a more accurate 3D model of the head 41.

[0140] (14) The program causes a computer to execute an image acquisition process for acquiring multiple images of the head 41 of an infant 4 taken from different angles; a filtering process for determining whether the images acquired in the image acquisition process are suitable images that satisfy predetermined conditions; and a three-dimensional model generation process for generating a three-dimensional model showing the three-dimensional external shape of the head based on the suitable images determined to satisfy the predetermined conditions in the filtering process.

[0141] (15) The program causes a computer to execute a three-dimensional model generation process for generating a three-dimensional model together with a server 3 that generates a three-dimensional model showing the three-dimensional external shape of the head 41 of an infant or young child 4 based on multiple images of the head 41. The three-dimensional model generation process includes an image acquisition process for acquiring multiple images of the infant's head taken from different directions, a filtering process process for determining whether the images acquired in the image acquisition process are suitable images that satisfy specified conditions, and a transmission process for transmitting the suitable images determined to satisfy the specified conditions in the filtering process to the server 3.

[0142] (16) The three-dimensional model generation method is a method executed by a computer, and includes an image acquisition step of acquiring multiple images of the head 41 of an infant or young child 4 taken from different angles, a filtering processing step of determining whether the images acquired in the image acquisition step are suitable images that satisfy predetermined conditions, and a three-dimensional model generation step of generating a three-dimensional model showing the three-dimensional external shape of the head 41 based on the suitable images determined to satisfy the predetermined conditions in the filtering processing step. [Explanation of symbols]

[0143] 1. 3D head model generation system 2. Electronic devices 3 Server 4. Infants and toddlers (target group) 20 Processing section 21 Photography Department 41 Head 201 Image acquisition unit (terminal side image acquisition unit) 205 Filtering processing unit 303 3D model generation unit

Claims

1. A head 3D model generation system comprising: an electronic terminal having an imaging unit that images a subject's head; and a processing unit that processes an image of the head imaged by the imaging unit; and a server that can communicate with the electronic terminal, The processing unit of the electronic terminal an image acquisition unit that acquires a plurality of images of the head taken from different directions by the imaging unit; a filtering processing unit that determines whether the image acquired by the image acquisition unit is a suitable image that satisfies a predetermined condition, The server and a three-dimensional model generation unit that generates a three-dimensional model that indicates the three-dimensional external shape of the head based on a plurality of matching images that are determined by the filtering processing unit to satisfy the predetermined conditions.

2. The system for generating a three-dimensional head model according to claim 1 , wherein the predetermined conditions include a condition regarding the degree of blurring of the image.

3. The filtering processing unit performing a Fourier transform on the image acquired by the image acquisition unit to acquire transformed data; 3. The system for generating a three-dimensional head model according to claim 2, wherein the degree of blurring of the image is determined based on an average value of spatial frequencies in the converted data, excluding low spatial frequency components.

4. 2. The system for generating a three-dimensional head model according to claim 1, wherein the predetermined conditions include a condition regarding a distance between the imaging unit and the head when the imaging unit captures an image of the head.

5. the photographing unit photographs the head in a state where the head is covered with a sheet, a plurality of markers are formed at intervals on a surface of the portion of the seat that covers the head; the image acquisition unit acquires the image in which the marker is located within a detection area that is a predetermined range from a center as an image corresponding to the marker; The filtering processing unit 5. The head 3D model generation system according to claim 4, wherein the distance between the image capturing unit and the marker is determined based on position information indicating the position of the marker on the image corresponding to the marker and parameter information including a focal length set in the image capturing unit and a size of an image sensor.

6. The system for generating a three-dimensional head model according to claim 1 , wherein the predetermined conditions include a condition regarding an orientation of the imaging unit relative to the head when the imaging unit captures an image of the head.

7. the photographing unit photographs the head in a state where the head is covered with a sheet, a plurality of markers are formed at intervals on a surface of the portion of the seat that covers the head; the image acquisition unit acquires an image in which the marker is located within a detection area that is a predetermined range from a center as an image corresponding to the marker; The filtering processing unit 7. The head 3D model generation system according to claim 6, wherein the orientation of the image capturing unit relative to the head is determined based on position information indicating the position of the marker on the image corresponding to the marker and parameter information including a focal length set in the image capturing unit and a size of an image sensor.

8. the photographing unit photographs the head in a state where the head is covered with a sheet, a plurality of identification markers associated with position information of the head are formed at intervals on the surface of the portion of the sheet that covers the head, and a plurality of colored patterns are formed; the image acquisition unit acquires the image in which the identification marker is located within a detection area that is a predetermined range from the center as an image corresponding to the identification marker; the filtering processing unit determines whether the image corresponding to the identification marker acquired by the image acquisition unit is the matching image; 2. The head 3D model generation system according to claim 1, wherein the 3D model generation unit generates the 3D model based on the position information and the color pattern associated with the identification markers of the matching images of images corresponding to all of the predetermined identification markers.

9. The system for generating a three-dimensional head model according to claim 8 , wherein all of the color patterns formed on the surface of the sheet are different from each other.

10. 2. The head 3D model generation system according to claim 1, wherein the predetermined conditions include a condition regarding a distance between the imaging unit and the head when the imaging unit images the head, and a condition regarding an orientation of the imaging unit with respect to the head when the imaging unit images the head.

11. The system for generating a three-dimensional head model according to claim 10 , wherein the predetermined conditions further include a condition regarding the degree of blurring of the image.

12. 12. The head three-dimensional model generation system according to claim 11, wherein the filtering processing unit determines whether or not the image acquired by the image acquisition unit satisfies the condition regarding the degree of blur, and then determines whether or not the image determined to satisfy the condition regarding the degree of blur satisfies a condition regarding the distance between the imaging unit and the head when the imaging unit images the head, and a condition regarding the orientation of the imaging unit with respect to the head when the imaging unit images the head.

13. the server further includes a background removal unit that detects a detection target image including a head of a subject in the matching image and removes information other than the detection target image from the matching image; 2. The head 3D model generation system according to claim 1, wherein the 3D model generation unit generates a 3D model showing the three-dimensional external shape of the head based on the plurality of matching images from which information other than the detection target image has been removed by the background removal unit.

14. an image acquisition step of acquiring a plurality of images of the subject's head, each image taken from a different direction; a filtering process step of determining whether the image acquired in the image acquisition step is a suitable image that satisfies predetermined conditions; a three-dimensional model generation step for generating a three-dimensional model showing the three-dimensional external shape of the head based on a plurality of matching images determined to satisfy the predetermined conditions in the filtering processing step; and a program for causing a computer to execute this step.

15. A program for causing a computer to execute a three-dimensional model generation process for generating a three-dimensional model representing a three-dimensional external shape of a subject's head based on a plurality of images of the subject's head, together with a server for generating the three-dimensional model, The three-dimensional model generation process includes: an image acquisition step of acquiring a plurality of images of the subject's head, each image taken from a different direction; a filtering process step of determining whether the image acquired in the image acquisition step is a suitable image that satisfies predetermined conditions; a transmitting step of transmitting a plurality of matching images determined to satisfy a predetermined condition in the filtering processing step to the server.

16. 1. A computer-implemented method for generating a three-dimensional model, comprising: an image acquisition step of acquiring a plurality of images of the subject's head, each image taken from a different direction; a filtering process step of determining whether the image acquired in the image acquisition step is a suitable image that satisfies predetermined conditions; and a three-dimensional model generation step of generating a three-dimensional model showing the three-dimensional external shape of the head based on a plurality of matching images determined to satisfy the predetermined conditions in the filtering processing step.

Images