Model construction method and device, electronic equipment and medium
By collecting image and scanning the surface and interior of the object, virtual external and internal models are constructed, which solves the problem that virtual models in the prior art cannot accurately describe the internal structure of the object, and improves the accuracy and detail of the model.
Patent Information
- Application Number
- CN202310790514.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2024-12-31
AI Technical Summary
When converting objects into virtual models, the prior art can only be constructed based on external features, and cannot accurately describe the internal structure of the object, resulting in insufficient accuracy of the virtual model.
By collecting image on the surface and interior of the object, determining external and internal shape parameters, and performing dimension scanning in combination with millimeter wave antenna arrays, virtual external and internal models are constructed to generate virtual models with external and internal structural characteristics.
Improve the accuracy of converting objects into virtual models, making the virtual models more suitable for actual objects, and enhancing the authenticity and detailedness of the models.
Smart Images
Figure CN119229061A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of model building, and in particular to a model building method, device, electronic device and medium. Background Art
[0002] With the rapid development of computer vision technology, the reconstruction of 3D objects has become possible. By taking photos of the surface of an object from multiple angles, it is easy to obtain rich information about the surface of the object, and then use a series of computer processing methods to segment, calibrate and reconstruct the collected images, so as to realize the construction of a three-dimensional virtual model based on the two-dimensional image of the object, and to facilitate the use of the virtual model of the object to adjust the application of data. For example, VR products convert objects into virtual models by taking photos with a camera and display them in VR products. However, when converting objects into virtual models, the virtual model can only be constructed based on the external features of the object, so that the constructed virtual model cannot accurately describe the object. Summary of the invention
[0003] The main purpose of the present application is to provide a model building method, device, electronic device and medium, aiming to solve the technical problem of how to improve the accuracy of converting an object into a virtual model.
[0004] To achieve the above objectives, the present application provides a model building method, which comprises the following steps:
[0005] Capturing an image of a surface of a target object to obtain a first image;
[0006] Determining external shape parameters of the target object according to the first image, and constructing a virtual external model according to external size information of the target object and the external shape parameters;
[0007] Capturing an image of the interior of the target object to obtain a second image;
[0008] determining internal shape parameters of the target object according to the second image, and constructing a virtual internal model according to internal size information of the target object and the internal shape parameters;
[0009] A target virtual model is constructed according to the virtual external model and the virtual internal model.
[0010] Optionally, the step of capturing an image of the interior of the target object to obtain a second image includes:
[0011] determining, based on the first image, whether there is a window on the surface of the object through which the interior of the object can be viewed;
[0012] If the window exists, capturing an image of the interior of the target object according to the window to obtain an internal image;
[0013] If the internal structure exists in the internal image, the internal image is determined to be the second image.
[0014] Optionally, before the step of constructing a virtual internal model according to the internal size information and the internal shape parameters of the target object, the method further comprises:
[0015] A first antenna array corresponding to the internal shape parameters is determined, and a size scan of the internal structure of the target object is performed in the window according to the first antenna array to obtain internal size information of the target object.
[0016] Optionally, the step of performing a size scan on the internal structure of the target object at the window according to the first antenna array to obtain the internal size information of the target object includes:
[0017] If there are multiple windows, determine the second image corresponding to each window;
[0018] If there is a second image with which the images overlap in each of the second images, taking the second image with which the images overlap as the third image, and selecting one window from among all windows corresponding to the third image as the target window;
[0019] The internal structure of the target object is scanned in size in the target window according to the first antenna array to obtain internal size information of the target object.
[0020] Optionally, after the step of determining the second image corresponding to each of the windows, the method further comprises:
[0021] Traversing the second image corresponding to each of the windows in turn, and matching the second image corresponding to the traversed window with the second images corresponding to other windows;
[0022] If there is a matching image that matches the second image corresponding to the traversed window in the second images corresponding to other windows, the matching image and the second image corresponding to the traversed window are used as second images with overlapping images.
[0023] Optionally, the step of determining the external shape parameters of the target object according to the first image includes:
[0024] Identifying a main body of the target object in the first image, and determining an external structure boundary corresponding to the target object in the first image based on the main body;
[0025] The object shape of the target object is determined according to the external structure boundary, and the object shape is used as an external shape parameter of the target object.
[0026] Optionally, before the step of constructing a virtual external model according to the external dimension information of the target object and the external shape parameters, the method further comprises:
[0027] A second antenna array corresponding to the external shape parameter is determined, and a dimension scan is performed on the surface of the target object according to the second antenna array to obtain external dimension information of the target object.
[0028] In addition, to achieve the above objectives, the present application also provides a model building device, including:
[0029] A surface acquisition module, used to acquire an image of the surface of the target object to obtain a first image;
[0030] A first construction module, configured to determine external shape parameters of the target object according to the first image, and to construct a virtual external model according to external size information of the target object and the external shape parameters;
[0031] An internal acquisition module, used for acquiring an image of the interior of the target object to obtain a second image;
[0032] a second construction module, configured to determine internal shape parameters of the target object according to the second image, and to construct a virtual internal model according to internal size information of the target object and the internal shape parameters;
[0033] A generating module is used to construct a target virtual model according to the virtual external model and the virtual internal model.
[0034] The present application also provides an electronic device, which is a physical device, and includes: a memory, a processor, and a program of the model building method stored in the memory and executable on the processor. When the program of the model building method is executed by the processor, the steps of the model building method as described above can be implemented.
[0035] The present application also provides a readable storage medium, which is the computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the model building method. The program for implementing the model building method is executed by a processor to implement the steps of the above-mentioned model building method.
[0036] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned model building method when executed by a processor.
[0037] The technical solution of the present application is to obtain a first image by performing image acquisition on the surface of the target object, and to construct a virtual external model based on external shape parameters determined by the first image and external size information of the target object, and to obtain a second image by performing image acquisition on the inside of the target object, and to construct a virtual internal model based on internal shape parameters determined by the second image and internal size information of the target object, and to construct a target virtual model based on the virtual external model and the virtual internal model, so that the constructed target virtual model has both external structural features and internal structural features of the target object, so that the constructed target virtual model is more closely fitted to the target object, thereby improving the accuracy of converting the object into a virtual model. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0039] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0040] Figure 1 This is a flow chart of the first embodiment of the model building method of the present application;
[0041] Figure 2 A schematic diagram of the scenario in the model building method of this application;
[0042] Figure 3 This is another schematic diagram of a scenario in the model building method of this application;
[0043] Figure 4 A schematic diagram of a millimeter wave antenna array in the model construction method of this application;
[0044] Figure 5 This is a schematic diagram of the overall process of the model building method of this application;
[0045] Figure 6 This is a flow chart of the second embodiment of the model building method of the present application;
[0046] Figure 7 A schematic diagram of the module for constructing the device for the model of this application;
[0047] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the electronic device in this application.
[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0049] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0050] In this embodiment, in order to improve the accuracy of converting objects into virtual models, the external and internal features of the model corresponding to the object are constructed based on the cooperation of the camera and millimeter wave antenna of the VR device, and the plane information of the model is improved by the camera, and the size information of the model is determined by the millimeter wave antenna. The internal structure of the model is constructed by identifying the windows in the model, adjusting the camera angle and changing the scanning accuracy of the millimeter wave antenna, thereby constructing a target virtual model with the surface and internal structure of the object.
[0051] Embodiment 1
[0052] Based on this, please refer to Figure 1 This embodiment provides a model building method, the model building method comprising:
[0053] Step S10, capturing an image of the surface of the target object to obtain a first image;
[0054] In this embodiment, the first image may be an image having surface features of the target object, and may be a plurality of images, such as an image set.
[0055] Optionally, the image acquisition device may be used to acquire images of the surface of the target object. The image acquisition device may be a device with an image acquisition function, such as a VR device, a camera, a mobile phone, etc. Optionally, when the image acquisition device is used to acquire images of the surface of the target object, the image acquisition reference coordinates may be determined first, and the image acquisition angle threshold and the image acquisition coordinates of each angle may be used to acquire images from multiple angles, and the camera parameters of each acquired image may be calibrated to obtain a calibration parameter set for each angle; by annotating the multi-angle acquired images and the calibration parameter sets for each angle in a one-to-one correspondence, an image set may be determined, and the image in the image set may be used as the first image. Optionally, the image set includes multiple images of the multi-angle image acquisition of the surface of the target object. The image acquisition reference coordinates may be the spatial coordinates of the camera in the VR device when the user is targeting the target object. The image acquisition angle threshold may be the angle threshold for multi-angle acquisition at 360 degrees around the target object. The image acquisition coordinates of each angle may be the spatial coordinates of the camera in the VR device at each image acquisition angle. The calibration parameter set for each angle includes the camera parameters at each image acquisition angle, such as focal length, camera rotation angle, and translation amount.
[0056] Step S20, determining the external shape parameters of the target object according to the first image, and constructing a virtual external model according to the external size information of the target object and the external shape parameters;
[0057] Optionally, after determining the first image or determining the image set, the appearance features of the target object may be determined based on the determined first image, so as to determine the external shape parameters of the target object according to the appearance features.
[0058] Optionally, all images in the image set can be determined and used as the first image, and then the first image can be subjected to image recognition processing. And when performing image recognition, image recognition can be performed on the first images corresponding to different angles in the image set to obtain image recognition results corresponding to each first image. Wherein, when performing image recognition, the target object in the first image can be identified, and the boundary contour information of the identified external structure boundary of the target object can be determined and used as the external shape parameter. For example, if the target object is a mobile phone, and the front shape of the mobile phone exists in the first image, the front shape of the mobile phone can be identified. Optionally, the image recognition result includes the boundary contour information of the external structure boundary of the target object. Wherein, the boundary contour information may include a contour edge line.
[0059] Optionally, the image recognition results corresponding to each first image can be summarized and screened to form a virtual model with a shape similar to the target object. Optionally, during the screening, the same part features in the image recognition results can be filtered and cleared. Optionally, the virtual model can be a three-dimensional model.
[0060] Optionally, all external dimension information of the target object, such as length, width, and height, is determined. Then, the virtual model constructed according to the external shape parameters is resized according to the external dimension information to obtain a virtual external model. It should be noted that the external dimension information needs to include dimension information corresponding to the external structure boundary of the target object. For example, Figure 2 The outer structural boundaries of the triangular pyramid shown may be the edges of the triangular pyramid and the edges of the window.
[0061] Optionally, the virtual external model corresponding to the target object may be determined and constructed in other ways, which are not limited here, such as constructing it according to a neural network model.
[0062] Step S30, capturing an image of the interior of the target object to obtain a second image;
[0063] Optionally, the interior of the target object needs to be detected to determine whether there is an internal structure, so the image acquisition device can be used to capture the interior of the target object to obtain a second image. Optionally, the method of capturing the interior of the target object is the same as the method of capturing the surface of the target object, except that the camera parameters are adjusted accordingly, such as adjusting the camera angle of the VR product, changing the camera focal length, etc.
[0064] Step S40, determining the internal shape parameters of the target object according to the second image, and constructing a virtual internal model according to the internal size information of the target object and the internal shape parameters;
[0065] Optionally, image recognition is performed on the second image to determine the internal shape parameters of the target object. When there are multiple second images, image recognition can be performed on each second image to obtain a recognition result of the image recognition. Optionally, the recognition result of the image recognition includes boundary contour information of the boundary of the internal structure of the target object, and the boundary contour information of the boundary of the internal structure of the target object is used as the internal shape parameter.
[0066] The recognition results of the image recognition corresponding to each second image are summarized and screened to form another virtual model having a shape similar to the internal structure of the target object. Optionally, during the screening, the same part features in the recognition results of the image recognition can be filtered and cleared. Optionally, the other virtual model can be a three-dimensional model.
[0067] Optionally, all internal dimension information of the target object, such as length, width, and height, is determined. Then, another virtual model constructed according to the internal shape parameters is resized according to the internal dimension information to obtain a virtual internal model. Optionally, the internal dimension information can be determined according to the millimeter wave antenna array, or according to all dimension information of the target object stored in advance. It should be noted that the internal dimension information needs to include the dimension information corresponding to the internal structure boundary of the target object. For example, Figure 2 The internal structure boundary of the triangular pyramid shown can be the circumference of the sphere in the window, or Figure 2 The circular edge corresponding to the sphere in the figure.
[0068] Step S50: constructing a target virtual model according to the virtual external model and the virtual internal model.
[0069] After obtaining the virtual external model and the virtual internal model, the two models can be aggregated and merged, such as embedding the virtual internal model into the virtual external model to obtain the target virtual model.
[0070] For example, Figure 2As shown, if the target object is a physical model with an internal structure and a window, its shape is a triangular cone, there is a square window on the triangular cone, and there is a spherical structure inside the triangular cone. The VR product first takes a photo of the physical model to obtain a first image, and performs image recognition on the first image to obtain the external structure boundary of the model. According to the external structure boundary of the model, a millimeter wave antenna array (that is, a millimeter wave array antenna) is selected to perform a rough scan of the physical model. For example, from the position of the external structure boundary of the model from left to right and from top to bottom, a scan is performed every 30 degrees. After the photo taking and scanning are completed, the shape of the model can be determined according to the first image, the external dimension information of the physical model can be determined by the scanning results, and the virtual external model of the physical model can be constructed according to the model shape (that is, the external shape parameters) and the external dimension information. Optionally, image recognition can be performed on the first image to identify the window position of the physical model, adjust the camera angle of the VR product, change the camera focal length, shoot the inside of the physical model, and obtain a second image. It is determined whether the physical model has an internal structure based on the second image. If there is an internal structure, adjust the millimeter wave antenna array to perform a fine scan of the inside of the physical model from the window position. For example, scan every 5 degrees from top to bottom and from left to right from the boundary of the internal structure. Determine the internal size information of the physical model based on the fine scanning results, determine the internal structure shape of the physical model (that is, the internal shape parameters) based on the second image, and construct the internal structure of the model based on the internal structure shape and internal size information of the physical model, that is, the virtual internal model. The target virtual model is constructed by combining the virtual external model and the virtual internal model, thereby completing the construction of the entire VR virtual model containing the internal structure model.
[0071] Optionally, if the physical model has multiple windows, such as Figure 3 As shown, it includes window 1, window 2, window 3 and window 4. After recognizing that the physical model has four windows according to the first image, adjust the camera angle of the VR product and take pictures of the inside of the model from the four windows respectively. If the images taken by window 1 and window 3, and window 2 and window 4 overlap, when the millimeter wave antenna array is used to measure the size, only window 1 and window 2 need to be scanned.
[0072] Alternatively, if Figure 4 As shown in the millimeter wave antenna array schematic, each dot or x represents an antenna in the array antenna. When scanning the external structure of the physical model, the dot antenna array in the array antenna is used to perform a rough scan on the outside of the physical model. When scanning the internal structure of the physical model, the x-numbered antenna array is used to perform a fine scan on the inside of the physical model.
[0073] In addition, to assist in understanding the model building process in this embodiment, an example is given below.
[0074] like Figure 5 As shown, start VR to take a photo of the model (that is, the target object), identify the model body through the photo, identify the external structure boundary of the model through the photo, determine the antenna array scanning range according to the boundary, determine the antenna array to be used, roughly scan the outside of the model (for example: scan 30 degrees from the model boundary and measure the size information), build the external structure of the virtual model, and determine whether the model has a window according to the photo. If not, it is determined that the virtual model has been built. If so, select the edge of the window, adjust the camera angle, zoom in and shoot the inside of the model, and determine whether the model has an internal structure according to the photo. If not, it is determined that the virtual model has been built. If so, determine the edge of the internal structure according to the photo, confirm the appropriate antenna array, scan the inside of the model in detail (for example: scan 5 degrees from the window boundary and measure the size information), build the internal structure of the virtual model, and build the virtual model by combining the internal structure of the virtual model and the external structure of the virtual model. The virtual model is built.
[0075] In this embodiment, a first image is obtained by performing image acquisition on the surface of the target object, and a virtual external model is constructed based on external shape parameters determined by the first image and external size information of the target object. A second image is obtained by performing image acquisition on the interior of the target object, and a virtual internal model is constructed based on internal shape parameters determined by the second image and internal size information of the target object. A target virtual model is constructed based on the virtual external model and the virtual internal model, so that the constructed target virtual model has both external structural features and internal structural features of the target object, so that the constructed target virtual model is more closely fitted to the target object, thereby improving the accuracy of converting the object into a virtual model.
[0076] Embodiment 2
[0077] Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 6 Step S30, capturing an image of the interior of the target object to obtain a second image, comprises:
[0078] Step a, determining whether there is a window on the surface of the object through which the interior of the object can be viewed according to the first image;
[0079] In this embodiment, when constructing the target virtual model corresponding to the target object, it is necessary to construct a model of the internal structure of the target object. Therefore, after acquiring the first images at different angles, image recognition can be performed on each first image to determine whether there is a window on the surface of the object in each first image. The user can view the internal features of the target object through the window.
[0080] Step b, if the window exists, collecting an image of the interior of the target object according to the window to obtain an internal image;
[0081] Optionally, if it is determined that there is a window on the surface of the target object through which the interior of the object can be viewed, the camera angle of the VR device can be adjusted, the camera focal length can be changed, and the interior of the target object can be photographed to complete image acquisition, and when performing image acquisition, different angles can be photographed according to the window to obtain multiple internal images. If there is no window, the virtual external model can be directly used as the target virtual model, and the target virtual model can be processed accordingly, such as displaying the target virtual model through a VR device.
[0082] Step c: if there is an internal structure in the internal image, determining the internal image to be a second image.
[0083] Optionally, each internal image is detected, and if there is at least one internal image with an internal structure image of the target object, an internal image with the most comprehensive internal structure image and the most features is selected from all internal images with internal structure images as the second image, and it is determined that the target object has structural features inside.
[0084] Optionally, if there is no image with an internal structure in all the internal images taken at the window, it can be determined that the target object has no internal structure, and the virtual external model can be directly used as the target virtual model, and the target virtual model can be processed accordingly, such as displaying the target virtual model through a VR device.
[0085] In this embodiment, when it is determined that there is a window on the surface of the object based on the first image, an image of the interior of the target object is captured based on the window to obtain an internal image, and when an internal structure is detected in the internal image, the internal image is determined to be the second image, thereby ensuring the validity of the acquired second image.
[0086] Furthermore, before the step of constructing a virtual internal model according to the internal size information and the internal shape parameters of the target object, the method further comprises:
[0087] Step d: determining a first antenna array corresponding to the internal shape parameters, and performing a size scan on the internal structure of the target object in the window according to the first antenna array to obtain internal size information of the target object.
[0088] In this embodiment, when constructing a virtual internal model of the target object, it is necessary to determine the internal size information of the target object. Therefore, the internal structure of the target object can be finely scanned by a millimeter wave antenna array to obtain the internal size information corresponding to the internal structure. For example, if the internal structure of the target object is a spherical object, the internal size information may include the diameter and circumference of the spherical object, and the position of the spherical object inside the target object.
[0089] Optionally, after determining the internal shape parameters of the target object, the required antenna array range can be determined according to the internal shape parameters, and an antenna array that is compatible with the internal shape parameters can be selected as the first antenna array in the antenna array range, and the internal structure of the target object can be finely scanned at the window according to the first antenna array to obtain internal size information corresponding to the internal shape parameters. When performing fine scanning, the scanning can be performed every 5 degrees from the internal structure boundary, from top to bottom, and from left to right.
[0090] In this embodiment, the internal size information is obtained by performing a size scan on the internal structure of the target object at the window according to the first antenna array corresponding to the internal shape parameters, thereby ensuring the accuracy of the obtained internal size information.
[0091] Furthermore, the step of performing a size scan on the internal structure of the target object at the window according to the first antenna array to obtain the internal size information of the target object includes:
[0092] Step e: if there are multiple windows, determine the second image corresponding to each window;
[0093] Optionally, when it is determined that there is a window in the target object, the camera angle of the VR product can be adjusted once, the focal length of the camera can be changed, and the interior of the target object can be photographed at the window to obtain the second image.
[0094] Optionally, when it is determined that there are multiple windows in the target object, each window can be photographed, that is, the camera angle of the VR product is adjusted multiple times to change the camera focal length. The internal images obtained by photographing each window are screened, and the second image is determined based on the internal image with the internal structure image.
[0095] Step f, if there is a second image with which the images overlap in each of the second images, taking the second image with which the images overlap as the third image, and selecting one window from among all windows corresponding to the third image as the target window;
[0096] Optionally, the second image corresponding to the window is an image obtained when the VR product is shooting in the window, and the image can display the internal structure of the target object.
[0097] Optionally, the second image corresponding to each window may be determined, and if the second image corresponding to a certain window is similar to the second image corresponding to another window, it may be determined that there is a second image with overlapping images. For example, if the second image corresponding to window A is similar to the second image corresponding to window B, it may be determined that the second image corresponding to window A and the second image corresponding to window B are second images with overlapping images, and they are used as the third image.
[0098] Then, a window is selected from all windows corresponding to the third image as the target window. For example, if the windows corresponding to the third image are window A and window B, a window can be selected from window A and window B as the target window, such as selecting window B as the target window.
[0099] Step g: performing a size scan of the internal structure of the target object in the target window according to the first antenna array to obtain internal size information of the target object.
[0100] Optionally, the internal structure of the target object can be finely scanned at the target window according to the first antenna array to obtain internal size information corresponding to the internal shape parameters. When performing fine scanning, the scan can be performed every 5 degrees from the internal structure boundary from top to bottom and from left to right.
[0101] For example, if there are three windows, namely window A, window B and window C, the images corresponding to window A and window B are the third image, and window B is selected as the target window, then the internal structure of the target object can be finely scanned in the target window and window C according to the first antenna array.
[0102] In this embodiment, when it is determined that there are multiple windows and there is a second image with overlapping images in the second image corresponding to each window, it is used as the third image, and a window is selected as the target window in the window corresponding to the third image. The internal structure of the target object is dimensionally scanned in the target window according to the first antenna array to obtain internal dimension information, thereby avoiding repeated scanning and achieving resource saving while obtaining the internal dimension information.
[0103] Further, after the step of determining the second image corresponding to each of the windows, the method further comprises:
[0104] Step h, traversing the second image corresponding to each of the windows in turn, and matching the second image corresponding to the traversed window with the second images corresponding to other windows;
[0105] In this embodiment, after determining the second image corresponding to each window, the second image corresponding to each window can be traversed, and the second image corresponding to the traversed window can be matched with the second image corresponding to other windows. And when matching, the internal structure image contained in the second image corresponding to the traversed window can be used as the first internal structure image, and the internal structure image contained in the second image corresponding to other windows can be used as the second internal structure image, and the similarity between the first internal structure image and each second internal structure image can be calculated. If there is a similarity greater than a preset threshold, the second internal structure image corresponding to the similarity greater than the preset threshold is used as the target internal structure image, and it is determined that the second image containing the target internal structure image matches the second image corresponding to the traversed window. If there is no similarity greater than the preset threshold, the second image corresponding to the next window continues to be traversed until the second images corresponding to all windows are traversed. Optionally, the other windows are windows other than the traversed windows among the multiple windows.
[0106] Step i: if there is a matching image that matches the second image corresponding to the traversed window in the second images corresponding to other windows, the matching image and the second image corresponding to the traversed window are used as the second images with overlapping images.
[0107] Optionally, if there is a second image that matches the second image corresponding to the traversed window, the matched second image is used as the matching image, and the matching image and the second image corresponding to the traversed window are used as second images with overlapping images.
[0108] In this embodiment, by traversing the second image corresponding to each window, and when there is a matching image that matches the second image corresponding to the traversed window in the second images corresponding to other windows, the matching image and the second image corresponding to the traversed window are used as the second images with overlapping images, thereby ensuring the accuracy of the second image with overlapping images.
[0109] Furthermore, the step of determining the external shape parameters of the target object according to the first image includes:
[0110] Step j, identifying the main body of the target object in the first image, and determining the external structure boundary corresponding to the target object in the first image based on the main body;
[0111] In this embodiment, after acquiring the first image with the surface features of the object, image recognition can be performed on the first image to identify the main body of the target object in the first image. The main body is the image of the target object in the first image. Then, the contour edge line of the main body in the first image is determined, and the contour edge line of the main body is used as the external structure boundary corresponding to the target object.
[0112] Step k: determining the object shape of the target object according to the external structure boundary, and using the object shape as the external shape parameter of the target object.
[0113] Optionally, if there are multiple first images, the external structure boundary corresponding to each first image may be determined, and the object shape of the target object, such as a triangular pyramid, a sphere, etc., may be constructed based on each external structure boundary. Optionally, the external shape parameter may include the object shape of the target object, and may also include boundary contour information of the identified external structure boundary of the target object.
[0114] Optionally, a virtual model similar in shape to the target object may be constructed based on each external structure boundary, so that the virtual model can be subsequently adjusted according to external dimension information to obtain a virtual external model.
[0115] In this embodiment, by identifying the main body of the target object in the first image, determining the external structure boundary based on the main body, and determining the object shape of the target object based on the external structure boundary, which are used as external shape parameters, the accuracy of the acquired external shape parameters is ensured.
[0116] Furthermore, before the step of constructing a virtual external model according to the external dimension information of the target object and the external shape parameters, the method further comprises:
[0117] Step x: determining a second antenna array corresponding to the external shape parameters, and performing a size scan on the surface of the target object according to the second antenna array to obtain external size information of the target object.
[0118] In this embodiment, when determining the external dimension information of the target object, the dimension information related to the target object input in advance by the user can be obtained and used as the external dimension information. Alternatively, the dimension information related to the target object stored in advance in the VR device can be used as the external dimension information. Alternatively, the surface of the target object can be dimensionally scanned by a millimeter wave antenna array. Before performing the dimension scan, the object shape of the target object can be determined based on the external shape parameters, and a suitable antenna array can be selected as the second antenna array based on the object shape, and then the surface of the target object can be dimensionally scanned based on the second antenna array to obtain the external dimension information of the target object.
[0119] In this embodiment, the external dimension information is obtained by performing a dimension scan on the surface of the target object according to the second antenna array corresponding to the external shape parameter, thereby ensuring the validity of the obtained external dimension information.
[0120] Embodiment 3
[0121] The present application also provides a model building device, please refer to Figure 7 , the model building device comprises:
[0122] The surface acquisition module A10 is used to acquire an image of the surface of the target object to obtain a first image;
[0123] A first construction module A20, configured to determine external shape parameters of the target object according to the first image, and to construct a virtual external model according to external size information of the target object and the external shape parameters;
[0124] An internal acquisition module A30 is used to acquire an image of the interior of the target object to obtain a second image;
[0125] A second construction module A40, configured to determine internal shape parameters of the target object according to the second image, and to construct a virtual internal model according to internal size information of the target object and the internal shape parameters;
[0126] The generating module A50 is used to construct a target virtual model according to the virtual external model and the virtual internal model.
[0127] Optionally, the internal acquisition module A30 is used to:
[0128] determining, based on the first image, whether there is a window on the surface of the object through which the interior of the object can be viewed;
[0129] If the window exists, capturing an image of the interior of the target object according to the window to obtain an internal image;
[0130] If the internal structure exists in the internal image, the internal image is determined to be the second image.
[0131] Optionally, the internal acquisition module A30 is used to:
[0132] A first antenna array corresponding to the internal shape parameters is determined, and a size scan of the internal structure of the target object is performed in the window according to the first antenna array to obtain internal size information of the target object.
[0133] Optionally, the internal acquisition module A30 is used to:
[0134] If there are multiple windows, determine the second image corresponding to each window;
[0135] If there is a second image with which the images overlap in each of the second images, taking the second image with which the images overlap as the third image, and selecting one window from among all windows corresponding to the third image as the target window;
[0136] The internal structure of the target object is scanned in size in the target window according to the first antenna array to obtain internal size information of the target object.
[0137] Optionally, the internal acquisition module A30 is used to:
[0138] Traversing the second image corresponding to each of the windows in turn, and matching the second image corresponding to the traversed window with the second images corresponding to other windows;
[0139] If there is a matching image that matches the second image corresponding to the traversed window in the second images corresponding to other windows, the matching image and the second image corresponding to the traversed window are used as second images with overlapping images.
[0140] Optionally, the first building module A20 is used to:
[0141] Identifying a main body of the target object in the first image, and determining an external structure boundary corresponding to the target object in the first image based on the main body;
[0142] The object shape of the target object is determined according to the external structure boundary, and the object shape is used as an external shape parameter of the target object.
[0143] Optionally, the first building module A20 is used to:
[0144] A second antenna array corresponding to the external shape parameter is determined, and a dimension scan is performed on the surface of the target object according to the second antenna array to obtain external dimension information of the target object.
[0145] The terminal provided by the embodiment of the present invention adopts the model building method of any one of the above-mentioned embodiments 1 to 2, which can improve the accuracy of converting an object into a virtual model. Compared with the prior art, the beneficial effects of the model building device provided by the embodiment of the present invention are the same as the beneficial effects of the model building method provided by the above-mentioned embodiment, and other technical features in the model building device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.
[0146] Embodiment 4
[0147] An embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model building method in the above-mentioned embodiment one.
[0148] Reference below Figure 8, which shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0149] like Figure 8 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0150] Typically, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.
[0151] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0152] The electronic device provided by the present invention adopts the model building method in the above embodiment to improve the safety of vehicle driving. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present invention are the same as the beneficial effects of the model building method provided by the above embodiment, and other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0153] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0154] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0155] Embodiment 5
[0156] An embodiment of the present invention provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the model building method in the above embodiment.
[0157] The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0158] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.
[0159] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device can execute each step in the above embodiment.
[0160] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0161] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0162] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.
[0163] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned model building method, which can improve the safety of vehicle driving. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present invention are the same as the beneficial effects of the model building method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.
[0164] Embodiment 6
[0165] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the above-mentioned model building method when executed by a processor.
[0166] The computer program product provided by the present application can improve the safety of vehicle driving. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the model building method provided by any of the above embodiments, which will not be described in detail here.
[0167] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.
Claims
1. A model building method, characterized in that: The model building method comprises the following steps: Capturing an image of a surface of a target object to obtain a first image; Determining external shape parameters of the target object according to the first image, and constructing a virtual external model according to external size information of the target object and the external shape parameters; Capturing an image of the interior of the target object to obtain a second image; determining internal shape parameters of the target object according to the second image, and constructing a virtual internal model according to internal size information of the target object and the internal shape parameters; A target virtual model is constructed according to the virtual external model and the virtual internal model.
2. The model building method according to claim 1, characterized in that: The step of capturing an image of the interior of the target object to obtain a second image comprises: determining, based on the first image, whether there is a window on the surface of the object through which the interior of the object can be viewed; If the window exists, capturing an image of the interior of the target object according to the window to obtain an internal image; If the internal structure exists in the internal image, the internal image is determined to be the second image.
3. The model building method according to claim 2, characterized in that: Before the step of constructing a virtual internal model according to the internal size information and the internal shape parameters of the target object, the method includes: A first antenna array corresponding to the internal shape parameters is determined, and a size scan of the internal structure of the target object is performed in the window according to the first antenna array to obtain internal size information of the target object.
4. The model building method according to claim 3, characterized in that: The step of performing a size scan on the internal structure of the target object at the window according to the first antenna array to obtain the internal size information of the target object comprises: If there are multiple windows, determine the second image corresponding to each window; If there is a second image with which the images overlap in each of the second images, taking the second image with which the images overlap as the third image, and selecting one window from among all windows corresponding to the third image as the target window; The internal structure of the target object is scanned in size in the target window according to the first antenna array to obtain internal size information of the target object.
5. The model building method according to claim 4, characterized in that: After the step of determining the second image corresponding to each of the windows, the method further comprises: Traversing the second image corresponding to each of the windows in turn, and matching the second image corresponding to the traversed window with the second images corresponding to other windows; If there is a matching image that matches the second image corresponding to the traversed window in the second images corresponding to other windows, the matching image and the second image corresponding to the traversed window are used as second images with overlapping images.
6. The model building method according to claim 1, characterized in that: The step of determining the external shape parameters of the target object according to the first image comprises: Identifying a main body of the target object in the first image, and determining an external structure boundary corresponding to the target object in the first image based on the main body; The object shape of the target object is determined according to the external structure boundary, and the object shape is used as an external shape parameter of the target object.
7. The model building method according to claim 1, characterized in that: Before the step of constructing a virtual external model according to the external dimension information and the external shape parameters of the target object, the method includes: A second antenna array corresponding to the external shape parameter is determined, and a dimension scan is performed on the surface of the target object according to the second antenna array to obtain external dimension information of the target object.
8. A model building device, characterized in that: The model building device comprises: A surface acquisition module, used to acquire an image of the surface of the target object to obtain a first image; A first construction module, configured to determine external shape parameters of the target object according to the first image, and to construct a virtual external model according to external size information of the target object and the external shape parameters; An internal acquisition module, used for acquiring an image of the interior of the target object to obtain a second image; a second construction module, configured to determine internal shape parameters of the target object according to the second image, and to construct a virtual internal model according to internal size information of the target object and the internal shape parameters; A generating module is used to construct a target virtual model according to the virtual external model and the virtual internal model.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the model building method according to any one of claims 1 to 7.
10. A medium, characterized in that The medium is a computer-readable storage medium, on which is stored a program for implementing the model building method, and the program for implementing the model building method is executed by a processor to implement the steps of the model building method as described in any one of claims 1 to 7.