Image determination method, device, electronic device and computer readable storage medium
By extracting image frames in a stationary state from multiple sets of video data, determining the synchronization frame group of each camera, and performing image modeling, the problem of low synchronization rate of multi-camera video frames is solved, improving the accuracy of image modeling and reducing costs.
Patent Information
- Application Number
- CN202111107699.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-22
- Publication Date
- 2025-03-07
- Estimated Expiration
- 2041-09-22
AI Technical Summary
During the three-dimensional image modeling process, the synchronization rate of video frames acquired by multiple cameras is low, which affects the accuracy of image modeling.
By acquiring multiple sets of video data, the video data is processed to obtain an image frame group, including image frames in which the target object is in a stationary state, thereby determining a synchronous frame group corresponding to each camera, and image modeling is performed based on these synchronous frame groups.
The synchronization rate of each synchronization frame is improved, thereby improving the accuracy of image modeling while reducing the cost of using a purely hardware-intensive camera array synchronization method.
Smart Images

Figure CN113763544B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and more particularly, to an image determination method, device, electronic device, and computer-readable storage medium. Background Art
[0002] With the development of 3D digital technology, 3D models are finding more and more applications in animation, gaming, virtual reality, 3D printing, and other fields. When using multiple cameras for data acquisition in 3D image modeling, it is crucial to synchronize the video frames captured by multiple cameras. Summary of the Invention
[0003] In view of this, embodiments of the present invention provide an image determination method, apparatus, electronic device, and computer-readable storage medium to improve the synchronization rate of each synchronization frame, thereby improving the accuracy of image modeling.
[0004] In a first aspect, an embodiment of the present invention provides an image determination method, the method comprising:
[0005] Acquire multiple sets of video data, where the multiple sets of video data are acquired through multiple cameras;
[0006] Processing the plurality of video data sets to obtain image frame groups corresponding to the respective video data sets, wherein the image frame groups include image frames in which the target object in the corresponding video data is in a stationary state;
[0007] Determining a synchronization frame group corresponding to each of the cameras from each of the image frame groups;
[0008] Image modeling is performed according to each of the synchronous frame groups to obtain a target image.
[0009] Optionally, the target object is periodically in a stationary state relative to each of the cameras;
[0010] The image frame group includes image frames corresponding to target objects in video data that are in respective static periods.
[0011] Optionally, determining the synchronization frame group corresponding to each camera from each image frame group includes:
[0012] An intermediate frame in each static period in each of the image frame groups is obtained to determine a synchronous frame group corresponding to each of the cameras.
[0013] Optionally, processing the multiple groups of video data to obtain image frame groups corresponding to the respective video data includes:
[0014] The plurality of video data sets are processed based on an optical flow method to obtain image frame sets corresponding to the respective video data sets.
[0015] Optionally, the plurality of cameras include a first camera and a second camera;
[0016] Performing image modeling according to each of the synchronous frame groups to obtain a target image includes:
[0017] Performing geometric reconstruction based on the synchronous frame group corresponding to the first camera to determine a geometric model corresponding to the target object;
[0018] The geometric model is texture mapped according to the synchronous frame group corresponding to the second camera to obtain the target image.
[0019] Optionally, multiple cameras are fixedly connected.
[0020] Optionally, the target object is placed on a rotating table, and the rotating table is controlled to remain stationary for a predetermined time after rotating a predetermined angle until it rotates one circle.
[0021] In a second aspect, an embodiment of the present invention provides an image determination device, the device comprising:
[0022] A video acquisition unit is configured to acquire multiple sets of video data, where the multiple sets of video data are acquired through multiple cameras;
[0023] a processing unit configured to process the plurality of video data sets to obtain image frame groups corresponding to the respective video data sets, wherein the image frame groups include image frames corresponding to the target object in the video data in a stationary state;
[0024] a synchronization frame acquisition unit, configured to determine a synchronization frame group corresponding to each of the cameras from each of the image frame groups;
[0025] The modeling unit is configured to perform image modeling according to each of the synchronous frame groups to obtain a target image.
[0026] Optionally, the target object is periodically in a stationary state relative to each of the cameras;
[0027] The image frame group includes image frames corresponding to target objects in video data that are in respective static periods.
[0028] Optionally, the synchronization frame acquisition unit includes:
[0029] The synchronous frame acquisition subunit is configured to acquire an intermediate frame in each static period in each of the image frame groups to determine the synchronous frame group corresponding to each of the cameras.
[0030] Optionally, the processing unit includes:
[0031] The processing subunit is configured to process the multiple groups of video data based on the optical flow method to obtain image frame groups corresponding to each video data.
[0032] Optionally, the plurality of cameras include a first camera and a second camera;
[0033] The modeling unit includes:
[0034] a geometric reconstruction subunit, configured to perform geometric reconstruction according to the synchronous frame group corresponding to the first camera, and determine a geometric model corresponding to the target object;
[0035] The texture mapping unit is configured to perform texture mapping on the geometric model according to the synchronous frame group corresponding to the second camera to obtain the target image.
[0036] Optionally, multiple cameras are fixedly connected.
[0037] Optionally, the target object is placed on a rotating table, and the rotating table is controlled to remain stationary for a predetermined time after rotating a predetermined angle until it rotates one circle.
[0038] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect of the embodiment of the present invention.
[0039] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method described in the first aspect of the embodiment of the present invention.
[0040] An embodiment of the present invention discloses an image determination method, device, electronic device and computer-readable storage medium, which acquire multiple groups of video data based on the acquisition of multiple cameras, process the multiple groups of video data to obtain image frame groups corresponding to each of the video data, determine the synchronization frame group corresponding to each of the cameras from each of the image frame groups, perform image modeling based on each of the synchronization frame groups, and obtain a target image, wherein the image frame group includes image frames in which the target object in the corresponding video data is in a stationary state, thereby improving the synchronization rate of each synchronization frame and thereby improving the accuracy of image modeling. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0042] Figure 1 is a schematic diagram of an image determination method according to an embodiment of the present invention;
[0043] Figure 2 Schematic diagram of the video acquisition process according to an embodiment of the present invention;
[0044] Figure 3 is a schematic diagram of a process for acquiring an image frame group according to an embodiment of the present invention;
[0045] Figure 4 1 is a schematic diagram of a process for obtaining a synchronization frame group according to an embodiment of the present invention;
[0046] Figure 5 is a schematic diagram of an image determination device according to an embodiment of the present invention;
[0047] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The present invention is described below based on the following embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, certain specific details are described in detail. Those skilled in the art can fully understand the present invention without these details. To avoid obscuring the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.
[0049] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.
[0050] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like in the specification should be interpreted as including rather than exclusive or exhaustive; that is, as “including but not limited to”.
[0051] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.
[0052] Figure 1 FIG is a schematic diagram of an image determination method according to an embodiment of the present invention. Figure 1 As shown, the image determination method according to the embodiment of the present invention includes the following steps:
[0053] Step S110: Acquire multiple sets of video data, wherein the multiple sets of video data are acquired through multiple cameras.
[0054] In an optional implementation, in this embodiment, multiple cameras simultaneously capture the target object to obtain multiple sets of video data. Optionally, the multiple cameras are fixedly connected to avoid relative movement of the cameras causing relative video capture offset, thereby preventing a decrease in synchronization rate.
[0055] In an optional implementation, when collecting video data of a target object, the target object is placed on a rotating table, and the rotating table is controlled to remain stationary for a predetermined time after each rotation of a predetermined angle until it rotates one circle, thereby making the target object periodically stationary relative to each camera.
[0056] Figure 2 FIG. 1 is a schematic diagram of the video acquisition process according to an embodiment of the present invention. Figure 2 As shown, this embodiment is described using two cameras as an example. In this embodiment, a rotating platform 21 is used to carry the target object 22. The rotating platform 21 is controlled to stop for a predetermined time after each rotation of a predetermined angle until it rotates one circle. Cameras 23 and 24 are rigidly fixedly connected, and two sets of video data are obtained by recording a video of the rotating platform 21 rotating one circle. Optionally, camera 23 can be a depth camera, and camera 24 can be a high-definition camera with a resolution greater than a predetermined value built into the smart device. Figure 2 As shown, cameras 23 and 24 are fixedly placed up and down. It should be understood that this embodiment does not limit the relative positions of cameras 23 and 24. After the relative positions of cameras 23 and 24 are fixed, the video data recorded by cameras 23 and 24 are made to correspond through camera calibration (i.e., adjusting the internal and external parameters of the cameras).
[0057] A depth camera, that is, a camera with depth detection function, can extract planar images, three-dimensional information, etc. of the target object. That is, each frame of image it scans includes a color RGB image of the points in the scene and the distance value of each point to the vertical plane where the depth camera is located.
[0058] Optionally, the predetermined time of the static period is greater than or equal to the exposure time of each camera. It should be understood that this embodiment does not limit the predetermined angle and predetermined time, which can be set according to the type of the target object.
[0059] Step S120: Process the multiple video data sets to obtain image frame groups corresponding to the respective video data sets, wherein the image frame groups include image frames corresponding to the target objects in the video data being in a stationary state.
[0060] In an optional implementation, as described above, the target object is periodically in a stationary state relative to each camera, and thus, the image frame group includes image frames corresponding to the target object in the video data in each stationary period.
[0061] In an optional implementation, step 120 may specifically include: processing multiple groups of video data based on an optical flow method to obtain image frame groups corresponding to each group of video data.
[0062] Optical flow is the instantaneous velocity of pixels moving across the observation plane of a moving object. It can be generated by the movement of the target object itself, the movement of the camera, or a combination of both. Optical flow uses the temporal changes in pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous and current frames, thereby calculating the motion information of objects between adjacent frames.
[0063] The basic constraint equation of the optical flow method is:
[0064] I x u+I y v+I t =0
[0065] Among them, I x , I y and I t They respectively represent the changes in the grayscale of the pixels in the image along the x-axis, y-axis, and t-axis (time axis), which can all be determined by the image data of adjacent frames. (u, v) is the optical flow vector.
[0066] The above basic constraint equation includes two unknown quantities u and v. Therefore, it is usually necessary to introduce other constraints, such as gradient (differential) based methods (such as Horn-Schunck algorithm, Lucas-Kanade algorithm, etc.), matching based methods, energy (frequency) based methods, phase based methods and neural dynamics based methods to calculate the optical flow vector.
[0067] Optionally, this embodiment may use an optical flow processing function (such as the Lucas-Kanade algorithm) in OpenCV (a computer vision library) to process video data. It should be understood that this embodiment does not limit the optical flow calculation method.
[0068] In the video data collected by the camera during the static period, adjacent frames are basically the same image. Therefore, the above-mentioned arbitrary optical flow method can be used to calculate the adjacent frames to be basically the same, that is, the grayscale of the pixel points is basically unchanged along the x-axis, y-axis and t-axis (time axis). The optical flow vector is basically adjacent frames to obtain an image frame group in which the target object in each group of video data is in a stationary state.
[0069] Figure 3 This is a schematic diagram of the acquisition process of the image frame group according to an embodiment of the present invention. Figure 2Take the video data obtained by cameras 23 and 24 as an example. Figure 3 As shown, the video data 31 includes video frames within the time period t0, time period t1, time period t2, and time period t3. Any of the above optical flow methods is used to calculate the changes in pixel points of adjacent frames in the video data 31 along the directions of each axis and the optical flow vector, thereby determining consecutive identical image frames to determine image frames in a static state. Figure 3 As shown, it can be determined that the target object is in a stationary state during time periods t1 and t3, and the video frames during time periods t1 and t3 are determined as image frame group 32. It should be understood that during the video acquisition process, the stationary period and the non-stationary period of the target object can be the same or different, and this embodiment is not limited to this.
[0070] Step S130: Determine the synchronization frame group corresponding to each camera from each image frame group. In one optional implementation, image frames at the same position in each image frame group are determined as corresponding synchronization frames. Alternatively, image frames at the same position within each static period in each image frame group are determined as corresponding synchronization frame groups. Alternatively, intermediate frames within each static period in each image frame group are obtained to determine the synchronization frame group corresponding to each camera.
[0071] Figure 4 This is a schematic diagram of the acquisition process of the synchronization frame group according to an embodiment of the present invention. Figure 3 The image frame group 32 shown in FIG. Figure 4 As shown, assuming that each image frame group includes N (N ≥ 1) static periods t1, t3, etc., the intermediate frame p1 within the first static period t1, the intermediate frame p2 within the second static period t3, and the intermediate frames within the other static periods in each image frame group are obtained to determine the corresponding synchronized frame group. Since the target object rotates during the video acquisition process, the synchronized frame group includes image frames of the target object at multiple angles.
[0072] In other optional implementations, image frames at other positions within the static period may be determined as synchronization frames. For example, the image frame at position 3 / 5 may be determined as the synchronization frame. That is, assuming that a static period includes 10 image frames, the 6th image frame may be determined as the synchronization frame.
[0073] In other optional implementations, consecutive predetermined frames located in the middle of the static period can be determined as synchronization frames. For example, three consecutive frames located in the middle can be determined as synchronization frames. Assuming that the static period includes 11 image frames, image frames 5-7 can be determined as synchronization frames. It should be understood that this embodiment does not limit the method for obtaining synchronization frames.
[0074] This embodiment can further improve the synchronization rate of the acquired video frames by determining the synchronization frames from the image frames in a static state, thereby improving the accuracy of image modeling.
[0075] Step S140: Perform image modeling based on each of the synchronized frame groups to obtain a target image. In an optional implementation, the multiple cameras include a first camera and a second camera. Optionally, the first camera is a depth camera, and the second camera is a high-definition camera with a resolution greater than a predetermined value. Step S140 may specifically include: performing geometric reconstruction based on the synchronized frame group corresponding to the first camera to determine a geometric model of the target object, and performing texture mapping on the geometric model based on the synchronized frame group corresponding to the second camera to obtain the target image.
[0076] In an optional implementation, feature points are extracted from the synchronous frames corresponding to the depth camera to determine the point cloud data of each synchronous frame, and the point cloud data of the synchronous frames at multiple angles are fused through the camera calibration parameters of the camera to obtain a geometric model.
[0077] Optionally, embodiments of the present invention further include determining intrinsic and extrinsic parameters of each camera. In this embodiment, any depth camera can be used to perform 3D reconstruction methods to adjust camera intrinsic and extrinsic parameters. Optionally, the Gauss-Newton method or alternating optimization method can be used to optimize camera intrinsic and extrinsic parameters. It should be understood that this embodiment does not limit the camera calibration method.
[0078] In this embodiment, the above-mentioned synchronization frame can be used to perform camera calibration to obtain the internal and external parameters of each camera, or separately taken images can be used, and this embodiment does not limit this. Optionally, this embodiment performs camera calibration by setting a calibration board to determine the internal and external parameters of each camera. In an optional implementation method, multiple cameras include a depth camera and a high-definition camera with a resolution greater than a predetermined value. A plurality of corresponding world coordinate three-dimensional points and corresponding image coordinate two-dimensional points can be obtained through the calibration board. The conversion of these three-dimensional points to two-dimensional points is calculated to determine the internal parameters of each camera, and the conversion of the camera coordinates of the depth camera to the camera coordinates of the high-definition camera (the conversion of the camera coordinates of the high-definition camera to the camera coordinates of the depth camera) is calculated to determine the camera extrinsic parameters from the depth camera to the high-definition camera (the camera extrinsic parameters from the high-definition camera to the depth camera). It should be understood that this embodiment does not limit the camera calibration method for multiple cameras.
[0079] In an optional implementation, based on the display status of the calibration plate, such as whether there is ghosting, the internal and external parameters of each camera determined by the above method are automatically fine-tuned or manually fine-tuned to avoid ghosting and other phenomena on the calibration plate as much as possible, thereby further improving the accuracy of the constructed geometric model and thus improving the clarity of the target image.
[0080] Texture is a high-level technology for modeling the surface properties of objects. The pixels in an image texture are often called texels. The core idea behind texture mapping is to establish a mapping method that matches a specific surface property of a fragment with data at a specific location in a data source. Then, through specific application methods, data obtained from the data source is used to change the value of a specific surface property, thereby affecting the shading calculation of the current fragment or other fragments, as well as the results of the merge output stage, resulting in a certain change in the appearance of the object's surface.
[0081] Optionally, in this embodiment, the three-dimensional points in the synchronized frame corresponding to the high-definition camera are converted into texture coordinates through a projection function, and projected into the parameter space to obtain a set of parameter space coordinates. One or more mapping functions are used to convert the parameter space coordinates into texture space positions, and the texture space positions are combined with the acquired geometric model to determine the target image.
[0082] The embodiment of the present invention acquires multiple sets of video data based on the acquisition of multiple cameras, processes the multiple sets of video data, obtains image frame groups corresponding to each of the video data, determines the synchronization frame groups corresponding to each of the cameras from each of the image frame groups, performs image modeling based on each of the synchronization frame groups, and obtains a target image, wherein the image frame group includes image frames in which the target object in the corresponding video data is in a stationary state. This improves the synchronization rate of each synchronization frame, thereby improving the accuracy of image modeling. At the same time, this embodiment can use a depth camera and a camera built into a smart device to more accurately synchronize video frames, which greatly reduces costs compared to a purely hardware-based dense camera array synchronization method.
[0083] Figure 5 FIG is a schematic diagram of an image determination device according to an embodiment of the present invention. Figure 5 As shown, the image determination device 5 according to the embodiment of the present invention includes a video acquisition unit 51 , a processing unit 52 , a synchronization frame acquisition unit 53 and a modeling unit 54 .
[0084] The video acquisition unit 51 is configured to acquire multiple sets of video data, each of which is acquired through multiple cameras. The processing unit 52 is configured to process the multiple sets of video data to obtain image frame groups corresponding to each of the video data, each of which includes image frames in which the target object in the corresponding video data is in a stationary state. The synchronization frame acquisition unit 53 is configured to determine the synchronization frame group corresponding to each of the cameras from each of the image frame groups. The modeling unit 54 is configured to perform image modeling based on each of the synchronization frame groups to obtain the target image. In an optional implementation, the multiple cameras are fixedly connected.
[0085] In an optional implementation, the target object is periodically in a stationary state relative to each of the cameras, and the image frame group includes image frames corresponding to the target object in the video data in each stationary period.
[0086] In one optional implementation, when capturing video data of a target object, the target object is placed on a rotating stage, and the rotating stage is controlled to remain stationary for a predetermined time after each predetermined angle rotation until it completes a full rotation, thereby causing the target object to periodically remain stationary relative to each camera. Alternatively, the target object is placed on a rotating stage, and the rotating stage is controlled to remain stationary for a predetermined time after each predetermined angle rotation until it completes a full rotation.
[0087] In an optional implementation, the synchronization frame acquisition unit includes a synchronization frame acquisition subunit configured to acquire an intermediate frame within each static period in each image frame group to determine a synchronization frame group corresponding to each camera.
[0088] In an optional implementation, the processing unit includes a processing sub-unit configured to process the plurality of video data sets based on an optical flow method to obtain image frame groups corresponding to each video data set.
[0089] In an optional implementation, the multiple cameras include a first camera and a second camera. The modeling unit includes a geometric reconstruction subunit and a texture mapping unit. The geometric reconstruction subunit is configured to perform geometric reconstruction based on the synchronized frame group corresponding to the first camera to determine a geometric model corresponding to the target object. The texture mapping unit is configured to perform texture mapping on the geometric model based on the synchronized frame group corresponding to the second camera to obtain the target image.
[0090] The embodiment of the present invention acquires multiple sets of video data based on the acquisition of multiple cameras, processes the multiple sets of video data, obtains image frame groups corresponding to each of the video data, determines the synchronization frame groups corresponding to each of the cameras from each of the image frame groups, performs image modeling based on each of the synchronization frame groups, and obtains a target image, wherein the image frame group includes image frames in which the target object in the corresponding video data is in a stationary state. This improves the synchronization rate of each synchronization frame, thereby improving the accuracy of image modeling. At the same time, this embodiment can use a depth camera and a camera built into a smart device to more accurately synchronize video frames, which greatly reduces costs compared to a purely hardware-based dense camera array synchronization method.
[0091] Figure 6 Schematic diagram of an electronic device according to an embodiment of the present invention. In this embodiment, the electronic device 6 includes a server, a terminal, etc. Figure 6As shown, the electronic device 6 includes at least one processor 61; a memory 62 connected to the at least one processor 61; and a communication component 63 connected to the scanning device, the communication component 63 receives and sends data under the control of the processor 61; wherein the memory 62 stores instructions that can be executed by the at least one processor 61, and the instructions are executed by the at least one processor 61 to implement the above-mentioned image determination method.
[0092] Specifically, the electronic device includes: one or more processors 61 and a memory 62, Figure 6 A processor 61 is taken as an example. The processor 61 and the memory 62 may be connected via a bus or other means. Figure 6 In this example, a bus connection is used. Memory 62, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. Processor 61 executes the non-volatile software programs, instructions, and modules stored in memory 62 to perform various functional applications and data processing of the device, thereby implementing the aforementioned image determination method.
[0093] The memory 62 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. Furthermore, the memory 62 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 62 may optionally include a memory remotely located relative to the processor 61, and such remote memory may be connected to an external device via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0094] One or more modules are stored in the memory 62 , and when executed by one or more processors 61 , perform the image determination method in any of the above method embodiments.
[0095] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.
[0096] The embodiment of the present invention acquires multiple sets of video data based on the acquisition of multiple cameras, processes the multiple sets of video data, obtains image frame groups corresponding to each of the video data, determines the synchronization frame groups corresponding to each of the cameras from each of the image frame groups, performs image modeling based on each of the synchronization frame groups, and obtains a target image, wherein the image frame group includes image frames in which the target object in the corresponding video data is in a stationary state. This improves the synchronization rate of each synchronization frame, thereby improving the accuracy of image modeling. At the same time, this embodiment can use a depth camera and a camera built into a smart device to more accurately synchronize video frames, which greatly reduces costs compared to a purely hardware-based dense camera array synchronization method.
[0097] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute part or all of the above method embodiments.
[0098] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by specifying relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.
[0099] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. An image determination method, characterized in that: The method comprises: Acquire multiple sets of video data, where the multiple sets of video data are acquired through multiple cameras, wherein the multiple cameras include a first camera and a second camera, the first camera is a depth camera, and the second camera is a high-definition camera; Processing the plurality of groups of video data based on an optical flow method to obtain image frame groups corresponding to the respective video data, wherein the image frame groups include image frames in which the target object in the corresponding video data is in a stationary state; Determine a synchronization frame group corresponding to each of the cameras from each of the image frame groups; Performing image modeling according to each of the synchronous frame groups to obtain a target image, wherein performing image modeling according to each of the synchronous frame groups to obtain a target image comprises: performing geometric reconstruction according to the synchronous frame group corresponding to the first camera to determine a geometric model of the target object, and performing texture mapping on the geometric model according to the synchronous frame group corresponding to the second camera to obtain the target image; The target object is periodically in a stationary state relative to each of the cameras, the image frame group includes image frames in which the target object in the corresponding video data is in each stationary period, and determining the synchronization frame group corresponding to each of the cameras from each of the image frame groups includes: The image frames at the same position in each static period in each of the image frame groups are determined as corresponding synchronous frame groups.
2. The method according to claim 1, characterized in that Determining the synchronization frame group corresponding to each of the cameras from each of the image frame groups includes: The intermediate frames in each static period in each of the image frame groups are obtained to determine the synchronous frame groups corresponding to each of the cameras.
3. The method according to claim 1, characterized in that The plurality of cameras are fixedly connected.
4. The method according to claim 1, characterized in that: The target object is arranged on a rotating platform, and the rotating platform is controlled to remain stationary for a predetermined time after rotating a predetermined angle until it rotates one circle.
5. An image determination device, characterized in that: The device comprises: A video acquisition unit is configured to acquire multiple sets of video data, where the multiple sets of video data are acquired through multiple cameras, wherein the multiple cameras include a first camera and a second camera, the first camera is a depth camera, and the second camera is a high-definition camera; A processing unit is configured to process the plurality of groups of video data based on an optical flow method to obtain image frame groups corresponding to the respective video data, wherein the image frame groups include image frames in which the target object in the corresponding video data is in a stationary state; A synchronization frame acquisition unit, configured to determine a synchronization frame group corresponding to each of the cameras from each of the image frame groups; A modeling unit, configured to perform image modeling according to each of the synchronous frame groups to obtain a target image; Wherein, the target object is periodically in a stationary state relative to each of the cameras; The image frame group includes image frames corresponding to the target object in the video data in each static period; The modeling unit includes: a geometry reconstruction subunit configured to perform geometry reconstruction according to the synchronous frame group corresponding to the first camera to determine the geometry model corresponding to the target object; a texture mapping unit configured to perform texture mapping on the geometry model according to the synchronous frame group corresponding to the second camera to obtain the target image; The synchronization frame acquisition unit is further configured to determine the image frames at the same position in each static period in each of the image frame groups as the corresponding synchronization frame group.
6. The device according to claim 5, characterized in that The synchronization frame acquisition unit comprises: The synchronous frame acquisition subunit is configured to acquire the intermediate frames in each static period in each of the image frame groups to determine the synchronous frame groups corresponding to each of the cameras.
7. The device according to claim 5, characterized in that The plurality of cameras are fixedly connected.
8. The device according to claim 5, characterized in that The target object is arranged on a rotating platform, and the rotating platform is controlled to remain stationary for a predetermined time after rotating a predetermined angle until it rotates one circle.
9. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 4.
10. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions, when executed by a processor, implement the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Real-time three-dimensional commodity imaging method and transaction system based on real-time three-dimensional display
CN107742318A