Dimensional measuring device, dimensional measuring method, and program

Abstract

A dimension measurement device (100) comprises a processor and memory. Using the memory, the processor acquires a three-dimensional model of a target object (S61), selects one of a plurality of basic shapes that are candidates for a three-dimensional shape (S62), fits the selected basic shape to the three-dimensional model (S63), and uses the fitted basic shape to measure the dimensions of the target object (S64). The dimension measurement device (100) may, for example, display display content for a user to designate one of the plurality of basic shapes at a display and select one of the plurality of basic shapes on the basis of information designated by the user via an input interface.

Description

Technical Field

[0001] The present disclosure relates to a dimensional measurement device, a dimensional measurement method, and a program.

Background Art

[0002] Techniques for measuring the dimensions of an object from a three-dimensional model such as point cloud data are known. For example, Patent Document 1 discloses a method for measuring dimensions using a point cloud obtained by a laser scanner.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In such measurements using a three-dimensional model, it is desired to improve the accuracy. An object of the present disclosure is to provide a dimensional measurement device, a dimensional measurement method, or a program that can improve the measurement accuracy.

Means for Solving the Problems

[0005] A dimensional measurement device according to an aspect of the present disclosure includes a processor and a memory. The processor uses the memory to acquire a three-dimensional model of an object, The display shows content on the screen for the user to specify the type of object, selects from a plurality of basic shapes that are candidates for a three-dimensional shape, , a basic shape pre-associated with a type specified by the user via the input interface fits the selected basic shape to the three-dimensional model, and measures the dimensions of the object using the basic shape after fitting.

Effects of the Invention

[0006] The present disclosure can provide a dimensional measurement device or a dimensional measurement method that can reduce the processing amount. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of dimensional measurement according to the embodiment. [Figure 2] Figure 2 shows an example of a screen displaying the dimensional measurement results according to the embodiment. [Figure 3] Figure 3 is a block diagram of a dimensional measuring device according to an embodiment. [Figure 4] Figure 4 is a block diagram of the imaging unit according to the embodiment. [Figure 5] Figure 5 is a block diagram of the control unit according to the embodiment. [Figure 6] Figure 6 is a block diagram of the dimension measuring unit according to the embodiment. [Figure 7] Figure 7 is a sequence diagram of the dimensional measurement process according to the embodiment. [Figure 8] Figure 8 shows an example of a three-dimensional model according to the embodiment. [Figure 9] Figure 9 shows an example of the correspondence between attributes and basic shapes according to the embodiment. [Figure 10] Figure 10 shows an example of the screen when selecting a basic shape according to the embodiment. [Figure 11] Figure 11 shows an example of the screen when selecting a basic shape according to the embodiment. [Figure 12] Figure 12 shows an example of attributes according to the embodiment. [Figure 13] Figure 13 is a schematic diagram showing the fitting process according to the embodiment. [Figure 14] Figure 14 shows an example of the basic shape after fitting according to the embodiment. [Figure 15] Figure 15 is a diagram illustrating the dimensional measurement process according to the embodiment. [Figure 16] Figure 16 is a flowchart of the dimensional measurement process according to the embodiment. [Figure 17] Figure 17 is a flowchart of the basic shape selection process according to the embodiment. [Figure 18] FIG. 18 is a flowchart of the dimension measurement process according to the embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0008] A dimension measuring apparatus according to an aspect of the present disclosure includes a processor and a memory. The processor uses the memory to acquire a three-dimensional model of an object, select one from a plurality of basic shapes that are candidates for the three-dimensional shape, fit the selected basic shape to the three-dimensional model, and measure the dimensions of the object using the basic shape after fitting.

[0009] According to this, the dimension measuring apparatus can improve the measurement accuracy by fitting a basic shape selected from a plurality of candidates for the three-dimensional shape of the object to the three-dimensional model and performing dimension measurement using the basic shape after fitting.

[0010] For example, the processor may cause a display to display display content for the user to specify one from the plurality of basic shapes, and the processor may select one from the plurality of basic shapes based on information specified by the user via an input interface.

[0011] According to this, the dimension measuring apparatus can select the basic shape to be fitted based on the user's selection, so that the accuracy of fitting can be easily improved. Therefore, the accuracy of dimension measurement using the basic shape after fitting can be improved.

[0012] For example, the display content is for the user to specify the three-dimensional shape of the object, and the processor may select the basic shape of the three-dimensional shape specified by the user via the input interface from the plurality of basic shapes.

[0013] According to this, by the user directly specifying the shape, a basic shape suitable for fitting with high accuracy can be selected.

[0014] For example, the display content is for the user to specify the type of object, and the processor may select a basic shape from the plurality of basic shapes that is pre-associated with the type specified by the user via the input interface.

[0015] According to this, users can make selections intuitively, thus improving user convenience.

[0016] For example, the processor may determine the type of object based on image recognition of the object's image, and the processor may select a basic shape from the plurality of basic shapes that is pre-associated with the determined type.

[0017] According to this, the dimensional measuring device can automatically select the basic shape without user input.

[0018] For example, the processor may select a basic shape from the plurality of basic shapes that is pre-associated with the location where the object or the dimensional measuring device is located.

[0019] According to this, the dimensional measuring device can automatically select the basic shape without user input.

[0020] For example, the processor may select one of the multiple basic shapes based on the result of fitting the three-dimensional model to each of the multiple basic shapes.

[0021] According to this, the dimensional measuring device can automatically select the basic shape without user input.

[0022] For example, the processor may determine the orientation of the object, and the processor may use the determined orientation to fit the three-dimensional model with the selected basic shape.

[0023] According to this, the dimensional measuring device can reduce the amount of fitting work required or improve accuracy.

[0024] Furthermore, a dimensional measurement method according to one aspect of this disclosure may involve acquiring a three-dimensional model of an object, selecting one of a plurality of basic shapes which are multiple candidates for the three-dimensional shape, fitting the selected basic shape to the three-dimensional model, and measuring the dimensions of the object using the fitted basic shape.

[0025] According to this, the dimensional measurement method can improve measurement accuracy by fitting a basic shape selected from multiple candidates for the three-dimensional shape of the object to a three-dimensional model, and then performing dimensional measurements using the fitted basic shape.

[0026] Furthermore, a program according to one aspect of this disclosure causes a computer to execute the dimensional measurement method.

[0027] These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium.

[0028] The embodiments will be described in detail below with reference to the drawings. Note that the embodiments described below are all specific examples of this disclosure. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, components in the following embodiments that are not described in an independent claim will be described as optional components.

[0029] (Embodiment) Efficiency can be improved by easily measuring the dimensions of items during collection. For example, this document describes a device and method for measuring the dimensions of an object from images captured by a camera on a mobile device such as a tablet or smartphone.

[0030] Figure 1 shows an example of dimension measurement in this embodiment. Figure 1 shows a user 11 (for example, a delivery worker or a delivery requester) using a dimension measuring device 100, which is a mobile terminal, to measure the dimensions of an object 10 (a golf bag in this example).

[0031] Figure 2 shows an example of the display screen of the dimension measuring device 100 in this case. For example, as shown in Figure 2, when the user photographs the object 10, the dimensions 12 of the object 10 (height, width, and depth in this example) are displayed.

[0032] Although the following primarily describes an example of measuring the dimensions of delivered items, the dimension measuring device 100 according to this embodiment can be applied to measuring the dimensions of any object. For example, the dimension measuring device 100 can be applied to measuring the dimensions of building structures at a construction site.

[0033] The configuration of the dimension measuring device 100 according to this embodiment will be described below. Figure 3 is a block diagram of the dimension measuring device 100. The dimension measuring device 100 comprises an imaging unit 200, a control unit 300, a dimension measuring unit 400, and a user interface 500.

[0034] The imaging unit 200 captures images (moving or still images). The control unit 300 controls the imaging unit 200, the dimension measurement unit 400, and the user interface 500. The dimension measurement unit 400 generates a three-dimensional model (e.g., point cloud data) by performing three-dimensional reconstruction using the images captured by the imaging unit 200. The dimension measurement unit 400 also selects one basic shape from a plurality of candidate three-dimensional shapes, fits the selected basic shape to the three-dimensional model, and measures the dimensions of the object using the fitted basic shape.

[0035] The user interface 500 accepts user input and presents information to the user. For example, the user interface 500 is a display and a touch panel. However, the user interface 500 is not limited to this and may be any user interface. For example, the user interface 500 may include at least one of the following: a keyboard, a mouse, a microphone, and a speaker.

[0036] Figure 4 is a block diagram showing the configuration of the imaging unit 200. The imaging unit 200 is, for example, a camera and comprises a storage unit 211, a control unit 212, an optical system 213, and an image sensor 214.

[0037] The memory unit 211 stores programs that the control unit 212 reads and executes. The memory unit 211 also temporarily stores video data of the imaging area captured using the image sensor 214, metadata such as timestamps attached to this video data, camera parameters of the imaging unit 200, and shooting settings such as the currently applied frame rate or resolution.

[0038] Such a storage unit 211 is implemented using, for example, a rewritable, non-volatile semiconductor memory such as flash memory. Depending on whether the stored data needs to be rewritten or the required storage period, a non-rewritable ROM (Read-Only Memory) or a volatile RaM (Random Access Memory) may also be used as the storage unit 211.

[0039] The control unit 212 is implemented, for example, using a CPU (Central Processing Unit), and controls each component of the imaging unit 200 to perform imaging and other functions by reading and executing a program stored in the memory unit 211. The control unit 212 may also be implemented by a dedicated circuit that controls each component of the imaging unit 200 to perform imaging and other functions. In other words, the control unit 212 may be implemented in software or in hardware.

[0040] The optical system 213 is a component that forms an image on the image sensor 214 from light from the imaging area, and is realized using optical elements including a lens. The focal length and angle of view of the optical system 213 may also be changeable. In addition, a wide-angle lens or an ultra-wide-angle lens such as a fisheye lens may be used.

[0041] The image sensor 214 is implemented as a solid-state image sensor such as a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, or a MOS (Metal-Oxide Semiconductor) image sensor, which receives light collected by the optical system on a light-receiving surface and converts this received light into electrical signals that represent an image.

[0042] Figure 5 is a block diagram showing the configuration of the control unit 300. The control unit 300 comprises an imaging control unit 301, a UI control unit 302, a dimension measurement control unit 303, and a storage unit 304.

[0043] The control unit 300 controls the imaging unit 200 and inputs and outputs signals to the user interface 500. The control unit 300 also issues commands to the dimension measurement unit 400 to measure dimensions based on the data received from the imaging unit 200.

[0044] Such a control unit 300 is implemented, for example, using a CPU. The storage unit 304 is implemented using a hard disk drive, various types of semiconductor memory, or a combination thereof. The storage unit 304 stores programs that the control unit 300 reads and executes. The storage unit 304 also stores data received from the imaging unit 200 that is subject to processing by the control unit 300.

[0045] The control unit 300 controls the imaging unit 200 and the dimension measurement unit 400 by reading and executing the program stored in the memory unit 304. The control unit 300 also executes processing in response to user commands related to these controls and processes.

[0046] Furthermore, one of these processes may include a dimensional measurement command. The UI control unit 302 is a functional component realized by the control unit 300 executing a program for obtaining instructions from the user. Similarly, the dimensional measurement control unit 303 is a functional component realized by the control unit 300 executing a program for dimensional measurement commands.

[0047] Furthermore, the imaging control unit 301, the UI control unit 302, and the dimension measurement control unit 303 may be implemented by dedicated circuits that perform imaging control, UI control, dimension measurement commands, and dimension measurement processing. In other words, the control unit 300 may be implemented in software or in hardware.

[0048] The imaging control unit 301, for example, causes the imaging unit 200 to image the three-dimensional space, which is the imaging area, at multiple different timings.

[0049] The UI control unit 302 sends imaging status information provided by the imaging unit 200 to the user interface 500 and obtains user input. User input includes the selection result of the data to be measured, whether or not to perform the dimension measurement process, or a combination thereof. If the user input is whether or not to perform the dimension measurement process, the UI control unit 302 outputs the feasibility of the dimension measurement process to, for example, the dimension measurement control unit 303.

[0050] The dimension measurement control unit 303 instructs the dimension measurement unit 400 to perform dimension measurement processing based on the feasibility of the dimension measurement process received from the UI control unit 302. Alternatively, the dimension measurement control unit 303 may instruct the dimension measurement unit 400 to perform dimension measurement processing based on the selection result of the data to be measured. Specific examples of processing by the dimension measurement control unit 303 will be described later.

[0051] Figure 6 is a block diagram showing the configuration of the dimension measurement unit 400. The dimension measurement unit 400 comprises an image acquisition unit 401, a preprocessing unit 402, a reconstruction unit 403, an attribute information extraction unit 404, a posture estimation unit 405, a fitting unit 406, and a measurement unit 407.

[0052] The dimension measurement unit 400 processes the data received via the control unit 300. Specifically, the dimension measurement unit 400 performs dimension measurement processing on objects located within a predetermined space captured by the imaging unit 200.

[0053] The image acquisition unit 401 acquires multiple images captured by the imaging unit 200. Each of the multiple images may be a still image or may consist of moving images.

[0054] The preprocessing unit 402 performs image preprocessing. Image preprocessing includes, for example, brightness adjustment, noise reduction, resolution conversion, color space conversion, lens distortion correction, projection conversion, affine conversion, edge enhancement processing, cropping processing, or a combination thereof. Image preprocessing may be performed in conjunction with the timing of the dimensional measurement processing, or it may be performed beforehand. Multiple preprocessed images obtained by performing image preprocessing by the preprocessing unit 402 may be stored in the storage unit 304 of the control unit 300. Note that each preprocessing step performed by the preprocessing unit 402 is not necessarily required. For this reason, the dimensional measurement unit 400 may be configured without a preprocessing unit 402.

[0055] The reconstruction unit 403 calculates the three-dimensional shape of a predetermined space using multiple images captured by the imaging unit 200. For example, the reconstruction unit 403 detects feature points in each of the multiple images captured by the imaging unit 200, uses the obtained feature points to establish correspondences between the images, and calculates a three-dimensional model of the predetermined space by geometric calculations based on the correspondences. For example, the three-dimensional model is point cloud data containing multiple three-dimensional points. Note that the three-dimensional model is not limited to a point cloud, but may also be a line cloud, a mesh, or a voxel.

[0056] The attribute information extraction unit 404 extracts attribute information from multiple images captured by the imaging unit 200 or from a three-dimensional model reconstructed by the reconstruction unit 403. Here, attribute information refers to information indicating the attribute (type) of the object to which a pixel in an image or a three-dimensional point in a three-dimensional model belongs. For example, the attribute information extraction unit 404 determines the attribute of an object in an image using known image recognition technology.

[0057] Furthermore, the attribute information extraction unit 404 may, for example, estimate the attributes of each point in a three-dimensional point cloud based on information of surrounding points, and calculate attribute information indicating the estimated attributes. Alternatively, the attribute information extraction unit 404 may, for example, estimate the attribute information of objects captured in each image based on information of surrounding pixels in image data, and link the estimated attribute information to each minimum constituent unit of the reconstructed three-dimensional model. The attribute information extraction unit 404 may also acquire the three-dimensional model reconstructed by the reconstruction unit 403. The attribute information extraction unit 404 may also process before the reconstruction unit 403 performs its processing. Furthermore, the reconstruction unit 403 may use the attribute information acquired by the attribute information extraction unit 404 to calculate only the three-dimensional shape of the region of an arbitrary attribute. Note that the attribute information extraction processing by the attribute information extraction unit 404 is not necessarily performed. For this reason, the dimension measurement unit 400 may have a configuration that does not include the attribute information extraction unit 404.

[0058] The posture estimation unit 405 estimates the posture of an object using multiple images captured by the imaging unit 200. For example, the posture estimation unit 405 estimates the posture on an image for an image region that shows an arbitrary attribute calculated by the attribute information extraction unit 404, and outputs the estimated posture as posture estimation information to the subsequent fitting unit 406. For example, if the object is cylindrical, the posture of the object can be determined by determining the long side and short side of the object in an image taken of the object from the side. Alternatively, posture information may be obtained in conjunction with the detection of object attributes using known image recognition techniques. The posture information may be a two-dimensional posture on the image or a three-dimensional posture.

[0059] Furthermore, in applications such as construction sites, the posture estimation unit 405 may determine the posture of the object using design information. Here, design information refers to CAD (Computer-Aided Design) data of the building structure, etc. In this case, for example, the shooting position and direction when the shooting device took the image are known, and the posture of the object being photographed in the image can be detected based on this information.

[0060] The fitting unit 406 performs a fitting process to apply the basic shape to the three-dimensional model reconstructed by the reconstruction unit 403.

[0061] The measurement unit 407 calculates the dimensions of the object using the basic shape after fitting. The object to be measured may be selected by the user or automatically selected. The calculation results are displayed on the user interface 500. Specifically, the measurement unit 407 uses the basic shape after fitting to calculate the distance between two faces of the basic shape, the size of the faces, or the length of the faces. For example, if the basic shape is a cylinder, the measurement unit 407 calculates the height of the cylinder by calculating the distance between the top and bottom faces, and outputs the calculated height of the cylinder and the diameter of the top surface of the cylinder as dimensions.

[0062] Figure 7 is a sequence diagram of the dimensional measurement process in the dimensional measuring device 100. First, the user issues a command to start shooting via the user interface 500 (S11). For example, the command to start shooting is given by selecting a menu on the screen or launching an application.

[0063] When the control unit 300 receives a start instruction, it sends an imaging instruction to the imaging unit 200. The imaging unit 200 captures multiple images (still images) according to the imaging instruction (S12). Here, the multiple images (still images) obtained are two or more images of the same object taken from different viewpoints. For example, the user takes images of an object (e.g., a golf bag) from different positions using a single imaging device (e.g., a tablet terminal).

[0064] The imaging unit 200 does not necessarily have to be included in the dimension measuring device 100; it may be included in a terminal other than the terminal containing the dimension measuring device 100. In this case, the image captured by the imaging unit 200 is sent to the dimension measuring device 100 via any communication means, such as wireless communication.

[0065] Multiple captured images are sent to the dimension measurement unit 400 via the control unit 300. The dimension measurement unit 400 generates a three-dimensional model by performing three-dimensional reconstruction using the multiple images (S13). The generated three-dimensional model is sent to the control unit 300. Figure 8 shows an example of a three-dimensional model. The three-dimensional model 20 is composed of multiple points, each having a three-dimensional coordinate. In this figure, the shape of the object (golf bag) is shown with a dashed line for reference, but the information shown by this dashed line is not included in the three-dimensional model 20. In addition, each point may have attribute information such as color and normal vector, as well as position information (e.g., three-dimensional coordinate).

[0066] Furthermore, the dimension measurement unit 400 generates attribute information indicating the attributes (type) of the object using multiple images (S14). For example, the attribute information extraction unit 404 determines the attributes of the object in the image using known image recognition technology. The generated attribute information is sent to the control unit 300.

[0067] Next, the control unit 300 outputs multiple candidate basic shapes to the user interface 500 to fit the three-dimensional model (S15). Here, the basic shape is a three-dimensional shape such as a rectangular parallelepiped, polygonal prism, polygonal pyramid, circle, cylinder, cone, or spherical crown. The basic shape may also be a shape represented by a combination of these. In other words, the basic shape is a three-dimensional shape composed of one or more planes or one or more curved surfaces, or a combination thereof.

[0068] Specifically, the control unit 300 uses attribute information to set priorities for a predetermined set of basic shape candidates. For example, one or more basic shape candidates are pre-associated with each of several attributes. Figure 9 shows an example of the correspondence between attributes and basic shape candidates. For example, as shown in Figure 9, a "cylinder" is associated with a "golf bag," and a "cuboid" is associated with a "cardboard box." The control unit 300 sets a higher priority for candidates associated with the attributes of the object.

[0069] Furthermore, if image recognition is used to determine attribute information, priority may be set based on the certainty obtained from the image recognition. Generally, image recognition calculates the certainty (probability) of multiple attributes for a single object. Therefore, the priority of the basic shape associated with the attribute with a higher certainty may be set higher. The determined candidates and priorities are sent to the user interface 500.

[0070] Next, the user selects a basic shape via the user interface 500 (S16). Figure 10 shows an example of this selection screen. For example, as shown in Figure 10, an image of the captured object 10 and several candidate basic shapes 30 are displayed. In Figure 10, a three-dimensional model is superimposed on the image, but the three-dimensional model does not have to be displayed. At this time, the display order of the multiple basic shapes 30 is determined according to the priority determined by the control unit 300. Specifically, basic shapes 30 with higher priority are displayed in higher positions so that they are more likely to be selected. Basic shapes with a priority lower than a predetermined threshold may be excluded, and only basic shapes with a priority higher than the threshold may be displayed.

[0071] Figure 11 shows another example of the selection screen. As shown in Figure 11, instead of the user selecting the basic shape itself, the user may select the attribute 31 (type) of the object 10. In this case as well, the display order of multiple attributes 31 may be determined according to the priority determined by the control unit 300. When an attribute is selected, the basic shape that is predefined and associated with that attribute is selected. The correspondence between the attribute and the basic shape is, for example, the same as in the example shown in Figure 9.

[0072] Furthermore, both attribute selection and basic shape selection may be combined. For example, in the screen shown in Figure 11, when an attribute is selected, one or more basic shapes that are predetermined to be associated with that attribute are displayed, and the user may select one of these one or more basic shapes.

[0073] Figure 12 shows an example of attributes. As shown in Figure 12, multiple attributes include cardboard boxes, paper bags, duffel bags, etc. In addition, one or more basic shapes are pre-associated with each attribute.

[0074] Furthermore, information on the selected basic shape is sent to the dimension measurement unit 400 via the control unit 300. The dimension measurement unit 400 then uses the captured image to estimate the orientation of the object (S17). Next, the dimension measurement unit 400 fits the selected basic shape to the three-dimensional model (S18).

[0075] Figure 13 is a diagram illustrating the fitting process. As shown in Figure 13, the dimension measurement unit 400 calculates the error between the three-dimensional model 20 and the basic shape 32 in each state while performing geometric transformations such as translation, rotation, and scaling in each axial direction on the selected basic shape 32. For example, the sum of the distances between each point in the three-dimensional model 20 and the basic shape 32 is calculated as the error. The dimension measurement unit 400 outputs the basic shape 32 in the state with the smallest calculated error as the fitted basic shape 33. Note that the sum of distances may exclude outliers or abnormal values.

[0076] Alternatively, the dimension measurement unit 400 may output the basic shape 32 in that state as the fitted basic shape 33 model when the calculated error becomes smaller than a predetermined threshold, and terminate the process. Figure 14 shows an example of the three-dimensional model 20 and the fitted basic shape 33.

[0077] Furthermore, the three-dimensional model 20 to be fitted may be the entirety of the generated three-dimensional model or only a part of it. This area of ​​the three-dimensional model may be selected by the user or automatically selected. For example, the dimension measurement unit 400 may select points belonging to the same attribute based on the extracted attribute information.

[0078] Furthermore, the estimated orientation of the object may be used as the initial value and / or constraint for this fitting. For example, the dimension measurement unit 400 may set the initial value of rotation to match the estimated orientation of the object. Alternatively, the dimension measurement unit 400 may limit the range of geometric transformation to a predetermined range from the estimated orientation of the object. This reduces the amount of computation and improves the accuracy of the fitting.

[0079] Next, the dimension measuring unit 400 measures the dimensions of the object 10 using the basic shape 33 after fitting (S19). Figure 15 shows an example of dimension measurement using the basic shape 33 after fitting. For example, as shown in Figure 15, the dimension measuring unit 400 measures the width, height, and depth of the object 10. For example, the measurement targets (width, height, and depth, etc.) may be predetermined for each basic shape or attribute. Alternatively, the measurement targets may be specified by the user. The obtained measurement results are sent to the control unit 300, which outputs the measurement results as dimension information to the user interface 500 (S20). Finally, the user interface 500 displays the dimensions 12 of the object 10 (golf bag), for example, as shown in Figure 2 (S21).

[0080] Figure 16 is a flowchart showing the processing flow of the dimension measurement unit 400. First, the dimension measurement unit 400 acquires multiple images of the object taken from different viewpoints (S41). Next, the dimension measurement unit 400 performs the above-described preprocessing on the acquired multiple images (S42). Next, the dimension measurement unit 400 generates a three-dimensional model by performing three-dimensional reconstruction using the multiple images after preprocessing (S43). The dimension measurement unit 400 also generates attribute information indicating the attributes (type) of the object using the multiple images (S44). For example, the attribute information extraction unit 404 determines the attributes of the object in the image using known image recognition technology.

[0081] Furthermore, the dimension measurement unit 400 uses the captured image to estimate the orientation of the object (S45). Note that the orientation estimation (S45) can be performed at any timing after step S42, as long as it is before the fitting process (S47).

[0082] Next, the dimension measurement unit 400 selects one of several basic shape candidates (S46). Figure 17 is a flowchart of this selection process. First, the dimension measurement unit 400 sets priorities for the several basic shape candidates based on the attribute information obtained by image recognition (S51). Next, the dimension measurement unit 400 displays the several basic shape candidates to the user based on the set priorities (S52). For example, the dimension measurement unit 400 displays several basic shape candidates in such a way that basic shape candidates with higher priorities are more likely to be selected by the user. Alternatively, the dimension measurement unit 400 displays only the candidates with a priority higher than a predetermined threshold from among the several basic shape candidates. Next, the dimension measurement unit 400 obtains information on the basic shape selected by the user from among the displayed several basic shape candidates (S53).

[0083] Next, the dimension measuring unit 400 fits the selected basic shape with the three-dimensional model (S47). Then, the dimension measuring unit 400 measures the dimensions of the object using the fitted basic shape (S48).

[0084] As described above, the dimension measuring device 100 according to this embodiment fits a basic shape to a three-dimensional model of an object and measures the dimensions of the object using the fitted basic shape. This makes it easy to measure the dimensions of any part of the object. For example, in the golf bag mentioned above, the dimensions excluding the handle can be measured accurately.

[0085] Furthermore, in this embodiment, the user selects the basic shape to be used for fitting from a plurality of candidate basic shapes. This improves the accuracy of fitting, and thus improves the accuracy of dimensional measurement. In addition, since the basic shape is selected by the user, the accuracy of the basic shape selection is improved, and thus the accuracy of fitting is improved.

[0086] Furthermore, by using image recognition or similar methods to set priorities for candidates, user convenience can be improved, and user selection errors can be reduced, thereby improving the accuracy of the fitting process.

[0087] The following describes modifications of this embodiment. The multiple processing units included in the dimensional measuring device 100 may be included in multiple devices. For example, the dimensional measuring device 100 may not include the imaging unit 200 and may acquire multiple images from an external device. The multiple images may be multiple images taken by multiple fixed cameras. The multiple images may also be images from two viewpoints taken by a stereo camera from a single position. The multiple images may also be multiple frames included in a moving image taken by a single camera. The multiple images may also be a combination of these.

[0088] Furthermore, the dimensional measuring device 100 may not include the reconstruction unit 403 and may acquire a three-dimensional model from an external device. Also, the three-dimensional model is not limited to one generated from multiple images; point cloud data obtained from a laser sensor such as LiDAR (Light Detection and Ranging) may be used. Additionally, some of the functions of the dimensional measuring device 100 may be included in a point cloud data generation device such as a laser sensor.

[0089] Furthermore, although the above explanation describes an example where the user selects the basic shape, the basic shape may be selected automatically. For example, the following method may be used.

[0090] (1) The dimensional measuring device 100 may select a basic shape that is pre-associated with the attributes of the object based on attribute information obtained from image recognition or the like.

[0091] (2) The dimension measuring device 100 may detect the position of the dimension measuring device 100 or the object and select a basic shape that is pre-associated with the detected position. The dimension measuring device 100 may be equipped with a position detection unit such as GPS (Global Positioning System). For example, if the position of the dimension measuring device 100 or the object is inside a warehouse, the dimension measuring device 100 may determine that the object is a box and select a rectangular parallelepiped as the basic shape. In other words, the dimension measuring device 100 may select a pre-defined basic shape when the position of the dimension measuring device 100 or the object is included in a pre-defined area. Note that multiple basic shapes may be associated with a single area. In other words, the dimension measuring device 100 may narrow down candidates from multiple basic shapes based on the position of the dimension measuring device 100 or the object. In this case, the basic shape to be used is selected from the multiple basic shapes associated with the area by user selection or other method.

[0092] (3) Methods other than image recognition may be used to determine attribute information. For example, when collecting a delivery item, information about the delivery item (including attribute information) may be registered in advance. In such cases, the attributes of the object may be determined based on that information. Furthermore, if the delivery item information includes the pickup address, the basic shape corresponding to the attribute information of the delivery item may be selected when the position information of the dimension measuring device 100 matches that address.

[0093] (4) The dimensional measuring device 100 may actually perform a fitting process for each of the multiple basic shapes and, based on the results, determine which basic shape to use in the end. Specifically, the dimensional measuring device 100 may calculate a fitting error for each basic shape, which is the minimum error between the three-dimensional model and the basic shape (i.e., the error between the three-dimensional model and the basic shape after fitting), and select the basic shape with the smallest fitting error among the multiple calculated fitting errors. Alternatively, if the minimum fitting error is less than a predetermined threshold, the dimensional measuring device 100 may select the basic shape with the smallest fitting error, and if the minimum fitting error is greater than or equal to the threshold, it may determine that there is no basic shape that fits the three-dimensional model and display a message indicating that there is no basic shape that fits the three-dimensional model. The error may also be the least squares error.

[0094] (5) The above methods may be combined. For example, the candidate basic shapes may be narrowed down using any of the methods (1) to (3) above, and the basic shape to be used in the end may be determined using the method (4) above.

[0095] (6) The priority of multiple candidates may be determined by any of the above methods, and then the user may make a selection.

[0096] (7) The user may set whether to automatically select the basic shape or to select it themselves. Alternatively, if automatic selection can narrow down the options to a single basic shape, automatic selection may be used, and if it is not possible to narrow down the options to a single basic shape, user selection may be used in combination. Alternatively, if accuracy cannot be obtained with automatic selection, it may be switched to user selection. For example, if the fitting error obtained with automatic selection is greater than a predetermined threshold, it may be switched to user selection.

[0097] (8) The dimension measuring device 100 may measure the dimensions of a cube (basic shape) that circumscribes the object when measuring dimensions. In this case, the dimension measuring device 100 may perform fitting such that the error resulting from fitting the part of the object that circumscribes the cube is given priority over the errors of other parts. For example, if the error is calculated as the sum of the distances between each point in the three-dimensional model and the basic shape, the distance corresponding to the part circumscribed to the three-dimensional object may be multiplied by a weighting coefficient that is larger than the distance corresponding to other parts.

[0098] As described above, the dimension measuring device 100 according to this embodiment performs the processing shown in Figure 18. The dimension measuring device 100 acquires a three-dimensional model of the object (S61), selects one of several basic shapes which are multiple candidates for the three-dimensional shape (S62), fits the selected basic shape to the three-dimensional model (S63), and measures the dimensions of the object using the fitted basic shape (S64). In this way, the dimension measuring device 100 can improve measurement accuracy by fitting a basic shape selected from multiple candidates for the three-dimensional shape of the object to the three-dimensional model and performing dimension measurement using the fitted basic shape.

[0099] For example, as shown in Figures 10 and 11, the dimensional measuring device 100 displays information on a display (e.g., a display included in the user interface 500) for the user to specify one of several basic shapes, and selects one of the several basic shapes based on the information specified by the user via an input interface (e.g., a touch panel, mouse, or keyboard included in the user interface 500). As a result, the dimensional measuring device 100 can select the basic shape to be fitted based on the user's selection, thus easily improving the accuracy of the fitting. Therefore, the accuracy of dimensional measurement using the basic shape after fitting can be improved.

[0100] For example, as shown in Figure 10, the displayed content is for the user to specify the three-dimensional shape of the object, and the dimension measuring device 100 selects a basic shape of the three-dimensional shape specified by the user via the input interface from a plurality of basic shapes. This allows the user to directly specify the shape and select a basic shape that is suitable for fitting with high accuracy.

[0101] For example, as shown in Figure 11, the displayed content is for the user to specify the type of object, and the dimension measuring device 100 selects a basic shape from a plurality of basic shapes that are pre-associated with the type specified by the user via the input interface. This allows the user to perform the selection operation intuitively, thereby improving user convenience.

[0102] For example, the dimensional measuring device 100 determines the type of object based on image recognition of the object's image, and selects a basic shape from a plurality of basic shapes that are pre-associated with the determined type. In this way, the dimensional measuring device 100 can automatically select a basic shape without user input.

[0103] For example, the dimension measuring device 100 selects a basic shape from a plurality of basic shapes that are pre-associated with the location of the object or the dimension measuring device. In this way, the dimension measuring device 100 can automatically select a basic shape without user input.

[0104] For example, the dimension measuring device 100 selects one of several basic shapes based on the result (e.g., error) of fitting a three-dimensional model to each of several basic shapes. In this way, the dimension measuring device 100 can automatically select a basic shape without user input.

[0105] For example, the dimension measuring device 100 determines the orientation of the object and uses the determined orientation to fit the three-dimensional model with the selected basic shape. For example, the dimension measuring device 100 uses the determined orientation as a constraint condition and / or an initial condition. This allows the dimension measuring device 100 to reduce the amount of fitting processing or improve accuracy.

[0106] For example, the dimensional measuring device 100 includes a processor and memory, and the processor uses the memory to perform the above processing.

[0107] The dimensional measuring device and the like according to the embodiments of this disclosure have been described above, but this disclosure is not limited to these embodiments.

[0108] Furthermore, each processing unit included in the dimensional measuring device, etc., according to the above embodiment is typically implemented as an integrated circuit, or as an LSI (Large Scale Integration). These may be individually integrated into a single chip, or some or all of them may be integrated into a single chip.

[0109] Furthermore, integrated circuit implementation is not limited to LSIs; it may also be achieved using dedicated circuits or general-purpose processors. Field-Programmable Gate Arrays (FPGAs), which can be programmed after LSI manufacturing, or reconfigurable processors, which allow for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used.

[0110] Furthermore, in each of the above embodiments, each component may be implemented by being composed of dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0111] Furthermore, this disclosure may be implemented as a dimensional measurement method, etc., performed by a dimensional measuring device, etc.

[0112] Furthermore, the division of functional blocks in the block diagram is just one example; multiple functional blocks can be implemented as a single functional block, a single functional block can be divided into multiple parts, or some functions can be moved to other functional blocks. In addition, the functions of multiple functional blocks with similar functions can be processed in parallel or time-sharing by a single piece of hardware or software.

[0113] Furthermore, the order in which each step in the flowchart is performed is illustrative for the purpose of specifically illustrating this disclosure, and may be in a different order. Also, some of the above steps may be performed simultaneously (in parallel) with other steps.

[0114] Although a dimensional measuring device, etc., relating to one or more embodiments has been described above based on embodiments, this disclosure is not limited to these embodiments. Without departing from the spirit of this disclosure, various modifications that a person skilled in the art could conceive of may be applied to these embodiments, and forms constructed by combining components from different embodiments may also be included within the scope of one or more embodiments. [Industrial applicability]

[0115] This disclosure is applicable to dimensional measuring devices. [Explanation of Symbols]

[0116] 10 Objects 11 users 12 dimensions 20 Three-Dimensional Models 30, 32 Basic shape 31 attributes 33 Basic shape after fitting 100 Dimensional measuring device 200 Imaging Unit 211 Storage section 212 Control Unit 213 Optical system 214 Image Sensor 300 Control Unit 301 Imaging Control Unit 302 UI Control Unit 303 Dimensional Measurement Control Unit 304 Storage section 400 Dimensional Measurement Section 401 Image acquisition unit 402 Pre-processing section 403 Reconstruction part 404 Attribute information extraction section 405 Posture estimation section 406 Fitting section 407 Measurement Unit 500 User Interfaces

Claims

1. Processor and Equipped with memory, The processor uses the memory to: Obtain a three-dimensional model of the object, The display shows content on the screen for the user to specify the type of object, From multiple basic shapes which are multiple candidates for three-dimensional shape, a basic shape that is pre-associated with a type specified by the user via the input interface is selected. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Dimensional measuring device.

2. A processor, Equipped with memory, The processor uses the memory to: Obtain a three-dimensional model of the object, Based on image recognition of the image of the object, the type of the object is determined. The basic shape that has been pre-associated with the determined type is selected from a plurality of basic shapes which are multiple candidates for three-dimensional shape. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Dimensional measuring device.

3. A dimensional measuring device, Processor and Equipped with memory, The processor uses the memory to: Obtain a three-dimensional model of the object, A basic shape that is pre-associated with the location where the object or the dimensional measuring device exists is selected from a plurality of basic shapes that are multiple candidates for the three-dimensional shape. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Dimensional measuring device.

4. The processor determines the orientation of the object, The processor uses the determined orientation to fit the three-dimensional model with the selected basic shape. A dimensional measuring device according to any one of claims 1 to 3.

5. Obtain a three-dimensional model of the object, The display shows content on the screen for the user to specify the type of object, From multiple basic shapes which are multiple candidates for three-dimensional shape, a basic shape that is pre-associated with a type specified by the user via the input interface is selected. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Method for measuring dimensions.

6. Obtain a three-dimensional model of the object, Based on image recognition of the image of the object, the type of the object is determined. The basic shape that has been pre-associated with the determined type is selected from a plurality of basic shapes which are multiple candidates for three-dimensional shape. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Method for measuring dimensions.

7. A method for measuring dimensions, Obtain a three-dimensional model of the object, A basic shape that is pre-associated with the object or the location where the dimensional measuring device for performing the dimensional measuring method is located is selected from a plurality of basic shapes which are multiple candidates for three-dimensional shapes. The selected basic shape is fitted to the three-dimensional model. The dimensions of the object are measured using the basic shape after fitting. Method for measuring dimensions.

8. To cause a computer to execute the dimension measurement method described in any one of claims 5 to 7. program.

Images