Scale determination method, device, electronic device and medium

Abstract

The present disclosure provides a scale determination method, device, electronic device and medium, which relate to the field of computer technology, and in particular to the field of computer vision technology. The specific implementation scheme is: obtain multiple images including the same target object and the same calibration block, and based on the multiple images, respectively reconstruct the target object and the calibration block in three dimensions to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block, and determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object. The present disclosure improves the certainty of the scale value between the size of the three-dimensional model obtained by three-dimensionally reconstructing an object using an image and the actual size of the object.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of computer vision technology. Background Art

[0002] Currently, objects can be reconstructed in three dimensions through multi-view red, green, and blue (RGB) images of objects. For example, in order to let buyers better understand the products, online shopping platforms can take photos of the products, obtain two-dimensional images of the products from multiple perspectives, and then reconstruct the products in three dimensions through these two-dimensional images, so as to show the three-dimensional model of the products to buyers. Summary of the invention

[0003] The present disclosure provides a scale determination method, device, electronic device, and medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a scale determination method is provided, including:

[0005] Acquire multiple images including the same target object and the same calibration block;

[0006] Based on the multiple images, three-dimensionally reconstruct the target object and the calibration block respectively to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block;

[0007] The scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block is determined as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0008] According to a second aspect of an embodiment of the present disclosure, there is provided a scale determination device, including:

[0009] An acquisition module, used for acquiring multiple images including the same target object and the same calibration block;

[0010] A reconstruction module, configured to perform three-dimensional reconstruction on the target object and the calibration block based on the multiple images, respectively, to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block;

[0011] The determination module is used to determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0012] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the method described in the first aspect when executed by a processor.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 is a flow chart of a scale determination method provided by an embodiment of the present disclosure;

[0021] Figure 2 is a schematic diagram of a method for photographing a target object and calibrating a target object provided by an embodiment of the present disclosure;

[0022] Figure 3 is a flow chart of another scale determination method provided by an embodiment of the present disclosure;

[0023] Figure 4 is a flow chart of another scale determination method provided by an embodiment of the present disclosure;

[0024] Figure 5 is an exemplary flow chart of a method for acquiring a true-size three-dimensional model of a target object provided by an embodiment of the present disclosure;

[0025] Figure 6 is a structural schematic diagram of a scale determination device provided by an embodiment of the present disclosure;

[0026] Figure 7 The block diagram is a block diagram of an electronic device for implementing the scale determination method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0029] It should be noted that the images involved in this embodiment are from a public dataset.

[0030] The scale value involved in the embodiments of the present disclosure is a fixed ratio scale, which may also be called a proportional scale or a geometric scale, and is used to represent the ratio between the three-dimensional model of a physical object and the physical object.

[0031] In the related art, it is necessary to perform three-dimensional reconstruction on some objects (such as goods on shopping websites). Currently, the RGB images of multiple perspectives of the object can be taken by a camera to reconstruct the three-dimensional model of the object. Since the distance between the camera and the object is different when the camera takes the object, the size of the object in the image is also different. For example, a small object is very close to the camera, while a large object is very far away from the object. In the image, the large object and the small object may appear to be the same size, so the scale value between the size of the object 3D model obtained based on the image reconstruction and the actual size of the object is uncertain.

[0032] In order to solve the problem that the scale of a three-dimensional model of an object obtained by using an RGB image of the object for three-dimensional reconstruction cannot be determined, the embodiments of the present disclosure provide a scale determination method, device, electronic device and medium.

[0033] The scale determination method provided in the embodiments of the present disclosure may be executed by an electronic device, which may be a smart phone, a tablet computer, a desktop computer, a server, or the like.

[0034] The following is a detailed description of the scale determination method provided by the embodiment of the present disclosure. Figure 1 As shown, the embodiment of the present disclosure provides a scale determination method, the method comprising:

[0035] S101, acquiring multiple images including the same target object and the same calibration block.

[0036] In the disclosed embodiment, the target object is an object that needs to be three-dimensionally reconstructed. For example, if applied to a scenario of an online shopping platform, the target object may be various commodities such as mobile phones, computers, shoes, etc. on the online shopping platform.

[0037] The calibration block may be a pre-selected object with measurable size. For the convenience of calculation, a symmetrical and simple object may be selected, such as a cube or a cuboid.

[0038] In order to accurately reconstruct the calibration block in three dimensions and meet the requirements of the three-dimensional reconstruction algorithm, a calibration block with rich surface textures, non-repetitive surface textures and no reflection can be selected.

[0039] Each image contains the same target object and the same calibration block, and each image is shot from a different viewing angle.

[0040] S102 . Based on the multiple images, three-dimensionally reconstruct the target object and the calibration block respectively to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block.

[0041] S103: Determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0042] Using the method of the disclosed embodiment, since multiple images all include the same calibration block and the same target object, the three-dimensional model of the calibration block and the three-dimensional model of the target object are both obtained by three-dimensional reconstruction based on these images, so the scale value of the three-dimensional model of the calibration block is the same as the scale value of the three-dimensional model of the target object. By calculating the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block, the scale value between the size of the three-dimensional model of the target object and the actual size of the target object can be obtained, thereby improving the certainty of the scale value between the size of the three-dimensional model obtained by three-dimensionally reconstructing the object using images and the actual size of the object.

[0043] In another embodiment of the present disclosure, the above S101 may be specifically implemented as follows:

[0044] Get multiple images taken by the camera around the target object and the calibration block for a specified number of circles; where the target object and the calibration block do not block each other from the camera's perspective.

[0045] In one embodiment, the target object can be photographed by an environmental system including a camera and a shooting box. A rotatable tray is placed in the shooting box, and the calibration block is fixed under the tray, and the target object is placed on the tray. The tray rotates during shooting. The target object on the tray and the calibration block under the tray rotate with the tray. The position of the camera can be fixed and aligned with the opening of the shooting box, and then images of the target object and the calibration block can be captured from multiple perspectives. It should be noted that the embodiment of the present disclosure does not limit the material of the calibration block. For example, if it is applied to the above-mentioned environmental system, a material with a certain hardness such as wood or hard plastic can be used, which can bear the weight of the tray and the object and will not deform.

[0046] Optionally, the rotation speed of the turntable and the shooting frequency of the camera can be set in advance, so that the camera can capture images of the target object and the calibration block at various angles during the rotation of the tray. The camera can adjust the placement angle of the target object once every time it completes a circle of shooting to capture images of the target object at various angles.

[0047] For example, Figure 2 As shown in the figure, assuming that the target object is a shoe, the shoe is placed on a tray in the shooting box, the calibration block is fixed under the tray, the camera is aimed at the opening of the shooting box, and the tray rotates so that the camera circles around the shoe and the calibration block to obtain multiple images of the shoe and the calibration block at different angles. Then adjust the placement angle of the shoe on the tray so that the shoe is exposed, and the tray rotates so that the camera circles around the shoe and the calibration block again to obtain multiple images of the calibration block and the shoe at different angles. After shooting two circles in total, images of the shoe at all angles can be obtained.

[0048] Accordingly, the electronic device can obtain multiple images including the calibration block and the shoes taken by the camera.

[0049] Alternatively, the target object and the calibration block may be placed as needed without relying on the shooting box, as long as the calibration block and the target object can be imaged in the same image and the target object does not block the calibration block. The target object and the calibration block may be photographed manually using a camera, or the camera may be placed in a circular track and moved around the target object and the calibration block to capture images of the target object and the calibration block at various angles.

[0050] With this method, since the calibration block and the target object will not block each other, the electronic device can completely and accurately reconstruct the three-dimensional model of the target object and the three-dimensional model of the calibration block. In addition, the multiple images acquired by the electronic device are obtained by the camera taking a specified number of circles around the target object and the calibration block, which is equivalent to capturing the target object and the calibration block in the same image during each shooting process of the camera, that is, the distance between the target object and the calibration block and the camera is the same, and the scale values ​​of the three-dimensional model of the target object and the three-dimensional model of the calibration block reconstructed based on these images are also the same. When the size of the calibration block is known, the scale value of the three-dimensional model of the calibration block can be accurately calculated, which is equivalent to obtaining the scale value of the three-dimensional model of the target object, thereby improving the certainty of the scale value of the three-dimensional model of the target object obtained by three-dimensional reconstruction.

[0051] In another embodiment of the present disclosure, Figure 3 As shown, based on the above embodiment, S102 can be specifically implemented as follows:

[0052] S1021 . Perform image segmentation on the target object and calibration block in each image to obtain multiple target object images and multiple calibration block images.

[0053] In the embodiment of the present disclosure, the electronic device may use an image segmentation method to segment each image into a target object image and a calibration block image.

[0054] For example, the image segmentation method adopted in the embodiment of the present disclosure may be a region segmentation method, a gray threshold segmentation method, an edge segmentation method, etc., and the embodiment of the present disclosure does not specifically limit this.

[0055] Taking the edge segmentation method as an example, the electronic device can perform edge detection on the image, detect the edge of the target object image and the edge of the calibration block image included in an image, and then segment the target object image and the calibration block image from the image.

[0056] S1022: Perform three-dimensional reconstruction of the target object based on the multiple target object images to obtain a three-dimensional model of the target object.

[0057] The electronic device may perform sparse reconstruction based on multiple target object images, for example, by using a Structure From Motion (SFM) method to perform sparse reconstruction to obtain a sparse point cloud. Other sparse reconstruction methods in the related art may also be used, which is not limited in the embodiments of the present disclosure.

[0058] After obtaining the sparse point cloud, the target object can be densely reconstructed based on the sparse point cloud to obtain the outline of the three-dimensional model of the target object, and then all the point clouds on the outline are gridded, and the point clouds in each grid are rendered differently, so that the grid point clouds in different areas present color, texture and detail characteristics close to the target object, thereby generating a three-dimensional model of the target object.

[0059] The embodiments of the present disclosure may also use other 3D reconstruction algorithms for 3D reconstruction, such as shape recovery from texture, shape recovery from shadow, and stereo vision algorithms, which are not specifically limited in the embodiments of the present disclosure.

[0060] S1023 . Perform three-dimensional reconstruction on the calibration block based on the multiple calibration block images to obtain a three-dimensional model of the calibration block.

[0061] The three-dimensional reconstruction process of the calibration block is the same as the method of reconstructing the three-dimensional model of the target object, and reference may be made to the relevant description in S1022 above.

[0062] By adopting the embodiment of the present disclosure, by performing image segmentation on the image containing the calibration block and the target object, an image containing the calibration block and an image containing the target object are obtained. Furthermore, in the three-dimensional reconstruction process of the calibration block and the target object, the calibration block image and the target object image will not interfere with each other, thereby improving the accuracy of the three-dimensional model and also improving the accuracy of the scale value.

[0063] In another embodiment of the present disclosure, based on the above embodiment, S103 can be specifically implemented in three ways, which are described below:

[0064] Method 1:

[0065] Gets the length of multiple specified dimensions of the 3D model of the calibration block.

[0066] For each specified dimension, the ratio of the length of the three-dimensional model of the calibration block in the specified dimension to the actual length of the calibration block in the specified dimension is calculated.

[0067] The average value of the ratios calculated for each specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0068] The electronic device can use the principal component analysis (PCA) method to obtain the principal axes of the three-dimensional model of the calibration block in multiple directions, and obtain the lengths D1, D2, D3, ... D of each dimension along the principal axes. iAlternatively, the electronic device can also find each plane of the three-dimensional model of the calibration block by plane fitting, find the edge by the intersection between the planes, and then calculate the length D1, D2, D3...D of each edge. i .

[0069] The actual length of each dimension of the calibration block is measured in advance using a measuring tool such as a tape measure, or other measuring tools with higher measurement accuracy can be used. For example, after measurement, the actual length of each dimension of the calibration block is d1, d2, d3…d i .

[0070] By calculation, we can get the scale value s=(∑ i D i / d i ) / i.

[0071] As an example, taking the calibration block as a cuboid, the three-dimensional model of the calibration block obtained by three-dimensional reconstruction is also a cuboid. The electronic device can obtain the lengths of three specified dimensions of the three-dimensional model of the calibration block, that is, the length, width and height of the three-dimensional model, and the actual length, width and height of the calibration block can be measured in advance.

[0072] Assume that the length, width and height of the three-dimensional model of the calibration block are D1, D2 and D3 respectively, and the actual length, width and height of the calibration block are d1, d2 and d3 respectively.

[0073] At this time, the scale value

[0074] Using this method, the ratio of the length of each specified dimension of the three-dimensional model of the calibration block to the length of the specified dimension of the calibration block can be calculated respectively, and the average value of multiple ratios can be calculated as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object, thereby reducing the error and making the obtained scale value more accurate.

[0075] Method 2:

[0076] The length of a specified dimension of the three-dimensional model of the calibration block is obtained, wherein the specified dimension can be specified according to actual needs, for example, the specified dimension can be the length of the longest side of the three-dimensional model, or the height of the three-dimensional model.

[0077] The ratio between the length of the three-dimensional model of the calibration block in the specified dimension and the actual length of the calibration block in the specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0078] The actual length of the calibration block in the specified dimension may be measured in advance, for example, the actual length of the specified dimension is d.

[0079] The electronic device may use a principal component analysis method or a plane fitting method or the like to obtain the length D of the three-dimensional model of the calibration block in the specified dimension.

[0080] Then the above scale value s=D / d.

[0081] As an example, taking the calibration block as a cuboid, the three-dimensional model of the calibration block obtained by three-dimensional reconstruction is also a cuboid. If the specified dimension is the height of the calibration block, the electronic device can obtain the height of the three-dimensional model of the calibration block.

[0082] Assume that the 3D model height of the calibration block is D3 and the actual height of the calibration block is d3.

[0083] The scale value

[0084] By adopting this method, only the ratio of the specified dimension of the three-dimensional model of the calibration block and the specified dimension of the calibration block can be calculated, thereby reducing the amount of calculation.

[0085] Method 3:

[0086] Get the maximum length of each dimension of the 3D model of the calibration block.

[0087] The ratio between the maximum value of the length of each dimension of the three-dimensional model of the calibration block and the maximum value of the length of each dimension of the calibration block is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0088] The length of each dimension of the calibration block can be measured in advance to obtain the maximum value d m .

[0089] After obtaining the three-dimensional model of the calibration block, the electronic device can use the principal component analysis method to obtain the length D of the dimension with the largest eigenvalue of the three-dimensional model of the calibration block. m .

[0090] At this time, the scale value

[0091] As an example, taking the calibration block as a rectangular block, the three-dimensional model of the calibration block obtained by three-dimensional reconstruction is also a rectangular block. At this time, the electronic device can use the principal component analysis method to obtain the length of the dimension with the largest eigenvalue corresponding to each dimension of the three-dimensional model of the calibration block.

[0092] Assume that each dimension of the three-dimensional model of the cuboid corresponds to the length D of the dimension with the largest eigenvalue m The actual length, width and height of the calibration block are the largest. m .

[0093] At this time, the scale value

[0094] In this way, the ratio between the maximum length of each dimension of the three-dimensional model of the calibration block and the maximum length of each dimension of the calibration block can be calculated, which can avoid inaccurate measurement of the actual length of the calibration block due to the selected length being too short, and avoid inaccurate size of the three-dimensional model calculated by PCA, which can improve the measurement and calculation accuracy, and thus improve the accuracy of the calculated scale value.

[0095] It should be noted that, in the above embodiment, the size of the calibration block can be measured once without repeated measurement.

[0096] In another embodiment of the present disclosure, if the embodiment of the present disclosure is applied to restore the true scale of the three-dimensional model of the target object, or is applied to the scene of establishing an accurate augmented reality (AR) model, then Figure 4 As shown, based on the above embodiment, after S103, the following steps are further included:

[0097] S104 . Based on a scale value between the size of the three-dimensional model of the target object and the actual size of the target object, transform the three-dimensional model of the target object into a three-dimensional model having the same size as the actual size of the target object.

[0098] In the disclosed embodiment, the electronic device can obtain the size of each dimension of the three-dimensional model of the target object, and divide the size of each dimension of the three-dimensional model of the target object by the scale value to obtain the actual size of each dimension of the three-dimensional model of the target object. The electronic device can then restore the size of each dimension of the three-dimensional model of the target object to the actual size to obtain a three-dimensional model with the same actual size as the target object.

[0099] For example, if the size of a certain dimension of the three-dimensional model of the target object is 5 cm and the scale value is 0.5, then the actual size of the dimension in the three-dimensional model of the target object is 5 / 0.5=10 cm.

[0100] By using the embodiments of the present disclosure, the actual size of each dimension of the three-dimensional model of the target object can be calculated based on the scale value and the three-dimensional model of the target object, and a three-dimensional model with the same actual size as the target object can be obtained, that is, a three-dimensional model that can restore the true scale of the target object.

[0101] Furthermore, the disclosed embodiments do not rely on structured light devices, red green blue depth map (RGBD) devices, rangefinders, scanners and other devices, do not require additional costs, are easy to operate, and can be suitable for application scenarios of three-dimensional reconstruction of goods or small objects.

[0102] The disclosed embodiments can also be applied to the scenario of measuring the size of a target object. After the scale value is obtained through the above embodiments, the actual size of the target object can be obtained based on the size and scale value of the three-dimensional model of the target object, thereby realizing the size measurement by three-dimensional reconstruction of the target object.

[0103] See also Figure 5 , Figure 5 This is an exemplary flow chart of a method for obtaining a real-size three-dimensional model of a target object provided in an embodiment of the present disclosure. Figure 5 Provide explanation.

[0104] S501, measuring the actual size of the calibration block.

[0105] S502: Image acquisition.

[0106] Place the calibration block and the target object together so that they do not block each other. Use a camera to capture the target object and the calibration block to obtain images from multiple angles.

[0107] The electronic device acquires images from multiple angles and segments each image to obtain multiple calibration block images and multiple target object images.

[0108] S503: Three-dimensional reconstruction of the calibration block.

[0109] The calibration block is three-dimensionally reconstructed according to multiple calibration block images.

[0110] S504: Obtain a three-dimensional model of the calibration block.

[0111] S505: 3D reconstruction of the target object.

[0112] The target object is reconstructed in three dimensions according to multiple target object images.

[0113] The execution order of S503 and S505 is not limited in the embodiment of the present disclosure. S503 and S505 may be executed in a sequential order or in parallel. Figure 5 The parallel execution of S503 and S505 is taken as an example.

[0114] S506: Acquire a three-dimensional model of the target object.

[0115] S507: Obtain the three-dimensional model scale value of the calibration block.

[0116] The electronic device uses principal component analysis to obtain the size of the three-dimensional model of the calibration block, and uses the scale value of the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0117] S508: Generate a real-size three-dimensional model of the target object.

[0118] The scale value between the size of the 3D model of the target object and the actual size of the target object is used to restore the size of each dimension of the 3D model of the target object to the actual size of each dimension of the target object, thereby obtaining a real-size 3D model of the target object.

[0119] This method does not require the use of additional equipment, saves costs, and is simple to operate. The scale value of the three-dimensional model of the target object can be accurately obtained. The scale value is applied to the three-dimensional model of the target object to obtain a three-dimensional model of the target object with a real scale.

[0120] Corresponding to the above method embodiment, the present disclosure embodiment also provides a scale determination device, such as Figure 6 As shown, the device comprises:

[0121] An acquisition module 601 is used to acquire multiple images including the same target object and the same calibration block;

[0122] A reconstruction module 602 is used to perform three-dimensional reconstruction on the target object and the calibration block based on multiple images to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block;

[0123] The determination module 603 is used to determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0124] In another embodiment of the present disclosure, the reconstruction module 602 is specifically configured to:

[0125] Performing image segmentation on the target object and calibration block in each image to obtain multiple target object images and multiple calibration block images;

[0126] Reconstructing the target object in three dimensions based on multiple target object images to obtain a three-dimensional model of the target object;

[0127] The calibration block is three-dimensionally reconstructed based on multiple calibration block images to obtain a three-dimensional model of the calibration block.

[0128] In another embodiment of the present disclosure, the determination module 603 is specifically configured to:

[0129] Obtain the lengths of multiple specified dimensions of the three-dimensional model of the calibration block;

[0130] For each specified dimension, calculating the ratio of the length of the three-dimensional model of the calibration block in the specified dimension to the actual length of the calibration block in the specified dimension;

[0131] The average value of the ratios calculated for each specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0132] In another embodiment of the present disclosure, the determination module 603 is specifically configured to:

[0133] Get the length of the specified dimension of the three-dimensional model of the calibration block;

[0134] The ratio between the length of the three-dimensional model of the calibration block in the specified dimension and the actual length of the calibration block in the specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0135] In another embodiment of the present disclosure, the acquisition module 601 is specifically used to acquire multiple images captured by the camera around the target object and the calibration block for a specified number of circles; wherein, in the camera's viewing angle, the target object and the calibration block do not block each other.

[0136] In another embodiment of the present disclosure, the device further includes:

[0137] The transformation module is used to transform the three-dimensional model of the target object into a three-dimensional model with the same size as the actual size of the target object based on the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

[0138] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0139] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0140] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0141] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0142] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the scale determination method. For example, in some embodiments, the scale determination method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the scale determination method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the scale determination method in any other appropriate manner (e.g., by means of firmware).

[0143] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0145] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0148] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0149] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0150] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A scale determination method, comprising: Acquire multiple images including the same target object and the same calibration block; Based on the multiple images, three-dimensionally reconstruct the target object and the calibration block respectively to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block; Determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object, wherein the scale value is used to represent the ratio between the three-dimensional model of the physical object and the physical object; The three-dimensional reconstruction of the target object and the calibration block is performed based on the multiple images to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block, including: Performing image segmentation on the target object and calibration block in each image to obtain multiple target object images and multiple calibration block images; Reconstructing the target object in three dimensions based on the multiple target object images to obtain a three-dimensional model of the target object; The calibration block is three-dimensionally reconstructed based on the multiple calibration block images to obtain a three-dimensional model of the calibration block.

2. The method according to claim 1, wherein: Determining the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object includes: Obtaining lengths of multiple specified dimensions of the three-dimensional model of the calibration block; For each specified dimension, calculating the ratio of the length of the three-dimensional model of the calibration block in the specified dimension to the actual length of the calibration block in the specified dimension; An average value of the ratios calculated for each designated dimension is used as a scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

3. The method according to claim 1, wherein: Determining the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object includes: Obtaining the length of a specified dimension of the three-dimensional model of the calibration block; The ratio between the length of the three-dimensional model of the calibration block in the specified dimension and the actual length of the calibration block in the specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

4. The method according to claim 1, wherein: The acquiring of multiple images of the same target object and the same calibration block comprises: Acquire multiple images captured by a camera around the target object and the calibration block for a specified number of circles; wherein, under the viewing angle of the camera, the target object and the calibration block do not block each other.

5. The method according to any one of claims 1 to 4, after determining the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object, the method further comprises: Based on a scale value between the size of the three-dimensional model of the target object and the actual size of the target object, the three-dimensional model of the target object is transformed into a three-dimensional model having the same size as the actual size of the target object.

6. A scale determination device, comprising: An acquisition module, used for acquiring multiple images including the same target object and the same calibration block; A reconstruction module, configured to perform three-dimensional reconstruction on the target object and the calibration block based on the multiple images, respectively, to obtain a three-dimensional model of the target object and a three-dimensional model of the calibration block; A determination module, used to determine the scale value between the size of the three-dimensional model of the calibration block and the actual size of the calibration block as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object, wherein the scale value is used to represent the ratio between the three-dimensional model of the physical object and the physical object; The reconstruction module is specifically used for: Performing image segmentation on the target object and calibration block in each image to obtain multiple target object images and multiple calibration block images; Reconstructing the target object in three dimensions based on the multiple target object images to obtain a three-dimensional model of the target object; The calibration block is three-dimensionally reconstructed based on the multiple calibration block images to obtain a three-dimensional model of the calibration block.

7. The device according to claim 6, wherein: The determination module is specifically used for: Obtaining lengths of multiple specified dimensions of the three-dimensional model of the calibration block; For each specified dimension, calculating the ratio of the length of the three-dimensional model of the calibration block in the specified dimension to the actual length of the calibration block in the specified dimension; An average value of the ratios calculated for each designated dimension is used as a scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

8. The device according to claim 6, wherein: The determination module is specifically used for: Obtaining the length of a specified dimension of the three-dimensional model of the calibration block; The ratio between the length of the three-dimensional model of the calibration block in the specified dimension and the actual length of the calibration block in the specified dimension is used as the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

9. The device according to claim 6, wherein: The acquisition module is specifically used to acquire multiple images captured by the camera around the target object and the calibration block for a specified number of circles; wherein, under the viewing angle of the camera, the target object and the calibration block do not block each other.

10. The device according to any one of claims 6 to 9, further comprising: The transformation module is used to transform the three-dimensional model of the target object into a three-dimensional model with the same size as the actual size of the target object based on the scale value between the size of the three-dimensional model of the target object and the actual size of the target object.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images