Scene modeling method, device and storage medium based on variable-focus monocular camera

Abstract

The present invention provides a scene modeling method based on a variable-focus monocular camera, comprising: controlling the monocular camera to zoom to a first zoom multiple at a current viewing angle, dividing the viewing image at the first zoom multiple into multiple viewing blocks; controlling the monocular camera to sequentially align with each of the multiple viewing blocks; when the monocular camera is aligned with any viewing block, controlling the monocular camera to zoom from the first zoom multiple to the second zoom multiple; determining the first moving distance of the feature point after zooming from the first zoom multiple to the second zoom multiple; determining the moving speed of the feature point according to the first moving distance; determining the depth value of the aligned viewing block according to the moving speed of the feature point, the second zoom multiple and a preset coefficient; controlling the monocular camera to rotate, and repeatedly performing the above steps; constructing a three-dimensional model according to the depth values ​​of the multiple viewing blocks at multiple viewing angles. The complexity, operation difficulty, monetary cost and time cost of scene modeling are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a scene modeling method, device and storage medium based on a variable-focus monocular camera. Background Art

[0002] The three-dimensional modeling technology of scenes has important applications in many fields, such as urban planning, virtual reality, geographic information systems, and environmental monitoring systems. These application scenarios usually require modeling of scenes within a radius of several kilometers, and different application scenarios have different requirements for modeling accuracy. At present, this type of large-scene three-dimensional modeling method usually relies on a multi-camera system or a laser radar system mounted on a movable platform (such as a vehicle or a drone). Although these methods can provide high-precision modeling results, they require professional equipment and complex algorithm support, and the complexity and operation difficulty are high, and the money cost is high. In addition, whether it is a multi-camera system or a laser radar system, its sampling distance is usually around tens of meters. Therefore, for modeling scenes within a few kilometers, it is necessary to drive a vehicle or a drone to traverse and collect the three-dimensional data of the scene. Since the modeling algorithm cannot be performed in real time, it is usually necessary to return to the problem location again to collect three-dimensional data after the problem is found in the later processing, and the time cost is very high. Therefore, how to reduce the complexity, operation difficulty, money cost and time cost of the three-dimensional modeling of the scene is a problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present invention provide a scene modeling method, device and storage medium based on a variable-focus monocular camera, aiming to reduce the complexity, operation difficulty, monetary cost and time cost of stereo modeling of the scene.

[0004] In a first aspect, an embodiment of the present invention provides a scene modeling method based on a variable-focus monocular camera, comprising:

[0005] Controlling a monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current viewing angle, and dividing a viewing image at the first zoom factor into a plurality of viewing blocks;

[0006] Controlling the monocular camera to sequentially align with each of the plurality of framing blocks;

[0007] When the monocular camera is aimed at any of the viewfinders, controlling the monocular camera to zoom from the first zoom factor to a second zoom factor, so that a feature point in the aimed viewfinder moves;

[0008] Determining a first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor;

[0009] determining a moving speed of the feature point according to the first moving distance;

[0010] Determine the depth value of the aligned view block according to the moving speed of the feature point, the second zoom factor and a preset coefficient, so as to obtain the depth values ​​of the multiple view blocks of the monocular camera at the view angle;

[0011] When the monocular camera fails to capture the scene to be modeled in a panoramic manner, controlling the monocular camera to rotate so as to change the viewing angle of the monocular camera;

[0012] Executing the step of controlling the monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current framing angle, and dividing the framing image at the first zoom factor into a plurality of framing blocks;

[0013] In the case that the monocular camera has captured a panoramic view of the scene to be modeled, a three-dimensional model of the scene to be modeled is constructed according to the depth values ​​of the multiple viewing blocks at the multiple viewing angles obtained.

[0014] In a second aspect, an embodiment of the present invention further provides a computer device, comprising a monocular camera with variable focus, a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the monocular camera, the processor and the memory, wherein when the computer program is executed by the processor, the scene modeling method as described in the first aspect is implemented.

[0015] In a third aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores a computer program, and the computer program can be executed by one or more processors to implement the scene modeling method as described in the first aspect.

[0016] The embodiments of the present invention provide a scene modeling method, device and storage medium based on a variable-focus monocular camera. The embodiments of the present invention only require the use of an inexpensive monocular camera to realize the construction of a three-dimensional model of the scene, and do not require a large number of multi-cameras or other 3D sensors, and do not require complex equipment deployment and professional operators, thereby effectively reducing the complexity, operational difficulty and monetary cost of stereoscopic modeling of the scene. The scene modeling method can be performed in real time, and the complexity of the algorithm is also low, which effectively reduces the time required for stereoscopic modeling of the scene, improves the efficiency of stereoscopic modeling of the scene and reduces the time cost of stereoscopic modeling of the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of a scene modeling method based on a variable-focus monocular camera provided by an embodiment of the present invention;

[0019] Figure 2 yes Figure 1 A schematic diagram of the sub-steps of the scenario modeling method in FIG.

[0020] Figure 3 is a flow chart of another scene modeling method provided by an embodiment of the present invention;

[0021] Figure 4 is a flow chart of another scene modeling method provided by an embodiment of the present invention;

[0022] Figure 5 It is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] The stereo modeling technology of ultra-large scenes has important applications in many fields, such as urban planning, virtual reality, geographic information systems, and environmental monitoring systems. These application scenarios usually require modeling of scenes within a radius of several kilometers, and different application scenarios have different requirements for modeling accuracy. At present, the stereo modeling methods of ultra-large scenes usually rely on multi-camera systems or lidar. Although these methods can provide high-precision modeling results, they require professional equipment and complex algorithm support, are complex and difficult to operate, and have high modeling costs.

[0027] To solve the above problems, an embodiment of the present invention provides a scene modeling method, device and storage medium based on a variable-focus monocular camera. The scene modeling method controls a monocular camera located in a scene to be modeled to zoom to a first zoom factor at a current viewing angle, then divides the viewing image at the first zoom factor into multiple viewing blocks, controls the monocular camera to sequentially align with each of the multiple viewing blocks, and when the monocular camera is aligned with any viewing block, controls the monocular camera to zoom from a first zoom factor to a second zoom factor, so that the feature points in the aligned viewing block move, and then determines the moving speed of the feature points according to the first moving distance of the pixel position of the feature points after zooming from the first zoom factor to the second zoom factor, and determines the depth value of the aligned viewing block according to the moving speed of the feature points, the second zoom factor and a preset coefficient, thereby obtaining the depth values ​​of the multiple viewing blocks of the monocular camera at the current viewing angle, and then when the monocular camera is not aligned with the first zoom factor, determines the depth value of the aligned viewing block. In the case of panoramic shooting of the scene to be modeled, the monocular camera is controlled to rotate to change the viewing angle of the monocular camera, and the above steps are repeated. In this way, when the monocular camera has panoramically shot the scene to be modeled, a three-dimensional model of the scene to be modeled is constructed according to the depth values ​​of multiple viewing blocks under the multiple viewing angles obtained, thereby realizing the three-dimensional model construction of the scene based on the monocular camera, without the need for a large number of multiple cameras or other 3D sensors, without the need for complex equipment deployment and professional operators, effectively reducing the complexity, operation difficulty and monetary cost of stereoscopic modeling of the scene, and the scene modeling method can be carried out in real time, and the complexity of the algorithm is also low, which effectively reduces the time required for stereoscopic modeling of the scene, improves the efficiency of stereoscopic modeling of the scene and reduces the time cost of stereoscopic modeling of the scene.

[0028] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 The present invention provides a flow chart of a scene modeling method based on a variable-focus monocular camera, which is applied to a computer device.

[0030] like Figure 1 As shown, the scene modeling method includes steps S101 to S108.

[0031] Step S101: Control a monocular camera located in a scene to be modeled to zoom to a first zoom factor at a current viewing angle, and divide a viewing image at the first zoom factor into a plurality of viewing blocks.

[0032] In this embodiment, the monocular camera is a zoomable monocular camera, and the scenes to be modeled may include a small scene with an area less than or equal to a first preset area, a medium scene with an area greater than the first preset area and less than or equal to a second preset area, a large scene with an area greater than the second preset area and less than or equal to a third preset area, and an extra-large scene with an area greater than the third preset area. Among them, the first preset area, the second preset area, and the third preset area can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this. For example, the first preset area is 50 square meters, the second preset area is 500 square meters, and the third preset area is 1000 square meters, that is, the coverage area is (0m 2 , 50m 2 ] is a small scene, covering an area of ​​(50m 2 , 500m 2 ] is a medium-sized scene, covering an area of ​​(500m 2 , 1000m 2 ] The scene in the range is a large scene, covering an area of ​​more than 1000m 2 The scene is a super large scene.

[0033] In some embodiments, dividing the framing image at the first zoom factor into a plurality of framing blocks includes: obtaining the modeling accuracy of the scene to be modeled, and determining the target number of rows and the target number of columns based on the modeling accuracy; dividing the framing image at the first zoom factor into framing blocks of the number of target rows*target number of columns. The target number of rows and the target number of columns can be determined based on the pre-established mapping relationship between the modeling accuracy, the number of rows and the number of columns, and the modeling accuracy of the scene to be modeled. For example, if the target number of rows is 30 and the target number of columns is 30, the framing image can be divided into 30*30=900 framing blocks. This embodiment divides the framing image into a corresponding number of framing blocks based on the modeling accuracy of the scene to be modeled, thereby ensuring that the accuracy of the finally constructed three-dimensional model meets the set modeling accuracy, so as to meet the different modeling accuracy requirements of the user for the three-dimensional model.

[0034] Step S102: Control the monocular camera to aim at each of the multiple view blocks in sequence.

[0035] In this embodiment, the monocular camera aligning the framing block may include aligning the monocular camera with the center position of the framing block, and the aligned framing block is located in the center area of ​​the framing image of the monocular camera, that is, the center position of the framing block is the same as the center position of the framing image of the monocular camera. The monocular camera can be controlled to rotate horizontally and / or vertically so that the monocular camera is aligned with the center position of the framing block, and the monocular camera can also be controlled to focus on the framing block, for example, the monocular camera is controlled to focus on the center position of the framing block so that the framing block is clear.

[0036] In some embodiments, the multiple viewing blocks include M rows and N columns of viewing blocks, and controlling the monocular camera to aim at each of the multiple viewing blocks in sequence may include: controlling the monocular camera to aim at each viewing block in each row in sequence from left to right or from right to left; or, controlling the monocular camera to aim at each viewing block in each odd row in sequence from left to right, and controlling the monocular camera to aim at each viewing block in each even row in sequence from right to left; or, controlling the monocular camera to aim at each viewing block in each odd row in sequence from right to left, and controlling the monocular camera to aim at each viewing block in each even row in sequence from left to right.

[0037] In some embodiments, the multiple viewing blocks include M rows and N columns of viewing blocks, and controlling the monocular camera to aim at each of the multiple viewing blocks in sequence may include: controlling the monocular camera to aim at each viewing block in each column in sequence from top to bottom or from bottom to top; or, controlling the monocular camera to aim at each viewing block in each odd-numbered column in sequence from top to bottom, and controlling the monocular camera to aim at each viewing block in each even-numbered column in sequence from bottom to top; or, controlling the monocular camera to aim at each viewing block in each odd-numbered column in sequence from bottom to top, and controlling the monocular camera to aim at each viewing block in each even-numbered column in sequence from top to bottom.

[0038] Step S103: when the monocular camera is aimed at any viewfinder, the monocular camera is controlled to zoom from a first zoom factor to a second zoom factor, so that a feature point in the aimed viewfinder moves.

[0039] In this embodiment, when the monocular camera is aimed at any viewfinder, the pixel position of the feature point in the aimed viewfinder changes with the change of the zoom factor of the monocular camera, so that the feature point in the aimed viewfinder moves with the change of the zoom factor. The pixel position of the feature point in the aimed viewfinder is the position of the feature point in the viewfinder.

[0040] In some embodiments, the feature point is determined based on a preset feature point extraction algorithm, wherein the preset feature point extraction algorithm includes a scale-invariant feature transform (SIFT) algorithm, a SURF (Speeded-Up Robust Features) algorithm, an ORB (Oriented FAST and Rotated BRIEF) algorithm, a Harris corner detection algorithm, or a Hessian-Laplace algorithm.

[0041] In some embodiments, the first zoom factor is different from the second zoom factor. Exemplarily, the first zoom factor is less than the second zoom factor or the first zoom factor is greater than the second zoom factor. For example, the first zoom factor is 1, and the second zoom factor is 2 or 5. Or the first zoom factor is 2 or 5, and the second zoom factor is 1. It is understandable that when the zoom factor changes, the moving speed of the feature points corresponding to the objects close to the monocular camera is faster than the moving speed of the feature points corresponding to the objects far from the monocular camera.

[0042] In some embodiments, controlling the monocular camera to zoom from a first zoom factor to a second zoom factor may include: according to a preset zoom rate, controlling the monocular camera to zoom from a first zoom factor to a second zoom factor. Among them, the preset zoom rate can be set based on actual conditions, and the embodiments of the present invention are not specifically limited to this. Exemplarily, the preset zoom rate is less than 0.5 times per second. For example, the preset zoom rate is 0.1 times per second, the first zoom factor is 1, and the second zoom factor is 2. Then, according to the zoom rate of 0.1 times per second, the monocular camera is controlled to increase the zoom factor from 1 times to 2 times, and the zoom time corresponding to the monocular camera increasing the zoom factor from 1 times to 2 times is 10 seconds.

[0043] Step S104: determining a first moving distance of a pixel position of the feature point after the pixel position is zoomed from the first zoom factor to the second zoom factor.

[0044] In this embodiment, the first moving distance is determined according to the first pixel position of the feature point in the framing block at the first zoom factor and the second pixel position of the feature point in the framing block at the second zoom factor. For example, the first pixel position of the feature point in the framing block at the first zoom factor is (x1, y1), and the second pixel position of the feature point in the framing block at the second zoom factor is (x2, y2), then the first moving distance is .

[0045] Step S105: Determine the moving speed of the feature point according to the first moving distance.

[0046] Exemplarily, the zoom time required for the monocular camera to zoom from a first zoom factor to a second zoom factor is obtained; the moving speed of the feature point is determined according to the zoom time required for the monocular camera to zoom from the first zoom factor to the second zoom factor and the first moving distance. Among them, the time when the monocular camera starts to zoom from the first zoom factor to the second zoom factor can be recorded by a timer to obtain the zoom time required for the monocular camera to zoom from the first zoom factor to the second zoom factor. If the monocular camera zooms from the first zoom factor to the second zoom factor according to a preset zoom speed, the zoom time required for the monocular camera to zoom from the first zoom factor to the second zoom factor can be determined according to the preset zoom speed and the difference between the first zoom factor and the second zoom factor.

[0047] Step S106: Determine the depth value of the aligned view block according to the moving speed of the feature point, the second zoom factor and the preset coefficient, so as to obtain the depth values ​​of the multiple view blocks under the view angle of the monocular camera.

[0048] In this embodiment, the preset coefficient is obtained through experiments, and the embodiment of the present invention does not specifically limit this. When the zoom ratio changes, the moving speed of the feature points corresponding to the objects close to the monocular camera is faster than the moving speed of the feature points corresponding to the objects far from the monocular camera. Therefore, the depth value of the viewfinder can be determined by the moving speed of the feature points, the zoom ratio and the preset coefficient.

[0049] In some embodiments, Figure 2 As shown, step S105 includes: sub-step S1051 to sub-step S1052.

[0050] Sub-step S1051: determining a depth value of the feature point according to the moving speed of the feature point, the second zoom factor and a preset coefficient.

[0051] Sub-step S1052: Determine the depth value of the aligned framing block according to the depth value of the feature point.

[0052] In this embodiment, when the zoom ratio changes, the moving speed of the feature point corresponding to the object close to the monocular camera is faster than the moving speed of the feature point corresponding to the object far from the monocular camera. In this way, the depth value of the feature point can be determined by the moving speed, zoom ratio and preset coefficient of the feature point, and the feature point is located in the viewfinder, so the depth value of the feature point can represent the depth value of the viewfinder, so the depth value of the viewfinder can be determined by the depth value of the feature point.

[0053] In some embodiments, determining the depth value of the feature point according to the moving speed of the feature point, the second zoom factor and the preset coefficient may include: determining the product of the moving speed of the feature point and the second zoom factor, and dividing the product by the preset coefficient to obtain the depth value of the feature point. The depth value of the feature point may be determined based on a calculation formula for the depth value, and the calculation formula for the depth value is: , D is the depth value of the feature point, V is the moving speed of the feature point, Z is the zoom factor, and K is the preset coefficient, which is obtained through experiments.

[0054] In some embodiments, determining the depth value of the aligned view block according to the depth value of the feature point may include: when there is one feature point, using the depth value of the feature point as the depth value of the aligned view block; when there are multiple feature points, calculating an average depth value according to the depth values ​​of the multiple feature points, and using the average depth value as the depth value of the aligned view block. In this embodiment, the accuracy of the depth value of the view block can be ensured by using the average of the depth values ​​of the multiple feature points as the depth value of the aligned view block.

[0055] It can be understood that, for each of the multiple viewing blocks at the current viewing angle, the following steps are repeatedly performed: controlling the monocular camera to aim at the viewing block; when the monocular camera is aimed at any viewing block, controlling the monocular camera to zoom from a first zoom factor to a second zoom factor so that the feature points in the aligned viewing block move; determining a first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor, and determining a moving speed of the feature point based on the first moving distance; determining a depth value of the aligned viewing block based on the moving speed of the feature point, the second zoom factor and a preset coefficient, thereby obtaining the depth values ​​of the multiple viewing blocks of the monocular camera at the viewing angle.

[0056] Step S107: When the monocular camera does not capture the scene to be modeled in a panoramic manner, the monocular camera is controlled to rotate to change the viewing angle of the monocular camera.

[0057] In this embodiment, when the depth values ​​of multiple viewing blocks of the monocular camera at the current viewing angle are obtained, it is determined that the depth value data of the monocular camera at the current viewing angle has been collected, and the depth value data at the next viewing angle can be collected. Therefore, when the monocular camera has not captured the scene to be modeled in a panoramic view, the monocular camera is controlled to rotate to change the viewing angle of the monocular camera, and then the process returns to execute steps S101 to S105 to obtain the depth values ​​of multiple viewing blocks of the monocular camera at the changed viewing angle, thereby obtaining the depth value data of the entire scene to be modeled.

[0058] In some embodiments, controlling the rotation of the monocular camera may include: controlling the monocular camera to rotate horizontally by a first angle and / or vertically by a second angle. The first angle and the second angle may be set based on actual conditions, and the embodiment of the present invention does not specifically limit this, and the first angle is different from the second angle. For example, the first angle is 30° and the second angle is 15°.

[0059] In some embodiments, the horizontal field of view of the monocular camera at 1x zoom is greater than or equal to 45°, and the vertical field of view of the monocular camera at 1x zoom is greater than or equal to 45°. The horizontal field of view and / or vertical field of view of the monocular camera decreases as the zoom factor increases, the horizontal rotation range of the monocular camera is 0 to 360°, and the vertical rotation range of the monocular camera is -60° to 60°, that is, the depression angle range of the monocular camera is -60 to 0°, and the elevation angle range of the monocular camera is 0° to 60°.

[0060] Step S108: When the monocular camera has captured the scene to be modeled in a panoramic manner, a three-dimensional model of the scene to be modeled is constructed according to the depth values ​​of multiple viewing blocks obtained at multiple viewing angles.

[0061] In this embodiment, when the monocular camera has captured a panoramic view of the scene to be modeled, the rotation angle of the monocular camera has reached a preset angle. Therefore, the depth values ​​of multiple view blocks at multiple view angles are the depth value data of the entire scene to be modeled. At this time, the view images of multiple view angles constitute a panoramic picture of the scene to be modeled. In this way, according to the depth values ​​of multiple view blocks at multiple view angles, a three-dimensional model of the entire scene to be modeled can be constructed. It can be understood that the preset angle can be set based on actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the preset angle is 360°, 300°, 270°, 240°, 180° or 120°, etc.

[0062] In some embodiments, according to the depth values ​​of multiple view blocks at multiple view angles, constructing a three-dimensional model of the scene to be modeled may include: obtaining a view image at each view angle, generating a depth map at each view angle according to the depth values ​​of multiple view blocks at each view angle; obtaining an internal parameter matrix and an external parameter matrix of a monocular camera at each view angle; and constructing a three-dimensional voxel model of the scene to be modeled according to the view image, the depth map, the internal parameter matrix and the external parameter matrix at each view angle. The modeling method provided by the embodiment of the present invention is relatively simple, and its calculation amount is much less than the modeling algorithm using a multi-camera system or a laser radar system. Therefore, a single computer device (desktop or industrial computer) containing a central processing unit (CPU) of i5 or above can meet the requirements of depth map generation and three-dimensional voxel model construction, without the need for graphics processing unit (GPU) computing power, and even more without the need to transmit the depth values ​​of multiple view blocks at multiple view angles to a server, and the server completes the construction of the model, which requires lower computing power for the device and has lower monetary cost.

[0063] In some embodiments, Figure 3 As shown, after step S106, the following steps are also included:

[0064] Step S201: construct a local three-dimensional model of the scene to be modeled at the viewing angle according to the depth values ​​of multiple viewing blocks of the monocular camera at the viewing angle. Then execute step S107, and return to execute steps S101 to S105.

[0065] Step S202: When the monocular camera has captured a panoramic view of the scene to be modeled, local three-dimensional models of the scene to be modeled obtained at multiple viewing angles are spliced ​​and merged to obtain a complete three-dimensional model of the scene to be modeled.

[0066] This embodiment constructs a local three-dimensional model of the scene to be modeled during the operation of the monocular camera, and then splices and fuses the local three-dimensional models under multiple viewing angles to obtain a complete three-dimensional model of the scene to be modeled, thereby improving the modeling speed of the scene to be modeled.

[0067] In some embodiments, Figure 4 As shown, after step S104, the following steps are also included:

[0068] Step S109: Determine whether the first moving distance is greater than or equal to a preset distance threshold.

[0069] In this embodiment, the preset distance threshold can be set based on actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the preset distance threshold is a distance of 2 or 5 pixels.

[0070] When the first moving distance is greater than or equal to the preset distance threshold, step S105 is executed to determine the moving speed of the feature point according to the first moving distance.

[0071] Step S110: When the first moving distance is less than a preset distance threshold, control the monocular camera to zoom to a third zoom factor, where the third zoom factor is greater than the second zoom factor.

[0072] In this embodiment, the monocular camera can be controlled to zoom from the second zoom factor to the third zoom factor, or the monocular camera can be controlled to retract from the second zoom factor to the first zoom factor, and then the monocular camera can be controlled to zoom from the first zoom factor to the third zoom factor.

[0073] Step S111: Determine a second moving distance of the feature point.

[0074] In this embodiment, the second moving distance of the feature point includes the moving distance of the feature point after the monocular camera is zoomed from the first zoom factor to the third zoom factor, or the moving distance of the feature point after the monocular camera is zoomed from the second zoom factor to the third zoom factor.

[0075] Step S112: when the second moving distance is greater than or equal to the preset distance threshold, determine the moving speed of the feature point according to the second moving distance, and determine the depth value of the aligned viewfinder according to the moving speed of the feature point, the third zoom factor and the preset coefficient.

[0076] This embodiment adjusts the zoom factor when the first moving distance of the feature point after the monocular camera is zoomed from the first zoom factor to the second zoom factor is less than a preset distance threshold, so that the moving distance of the feature point is larger (the movement of the feature point is more obvious), thereby ensuring the accuracy of the moving speed of the feature point and improving the accuracy of the depth value, thereby improving the modeling accuracy.

[0077] In some embodiments, determining the moving speed of the feature point based on the second moving distance may include: when the second moving distance is the moving distance of the feature point after the monocular camera zooms from the first zoom factor to the third zoom factor, determining the moving speed of the feature point based on the second moving distance and the zoom time required for the monocular camera to zoom from the first zoom factor to the third zoom factor; when the second moving distance is the moving distance of the feature point after the monocular camera zooms from the second zoom factor to the third zoom factor, determining the moving speed of the feature point based on the second moving distance and the zoom time required for the monocular camera to zoom from the second zoom factor to the third zoom factor.

[0078] See also Figure 5 , Figure 5It is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention.

[0079] like Figure 5 As shown, the computer device 100 includes a processor 101, a memory 102 and a monocular camera 103 with variable focus. The processor 101, the memory 102 and the monocular camera 103 are connected via a bus 104, which is, for example, an I2C (Inter-integrated Circuit) bus.

[0080] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire computer device. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0081] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0082] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the embodiment of the present invention, and does not constitute a limitation on the computer device to which the embodiment of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0083] The processor 101 is used to run a computer program stored in the memory 102, and implement any one of the scene modeling methods provided by the embodiments of the present invention when executing the computer program.

[0084] In one embodiment, the processor 101 is used to run a computer program stored in a memory, and implements the following steps when executing the computer program:

[0085] Controlling a monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current viewing angle, and dividing a viewing image at the first zoom factor into a plurality of viewing blocks;

[0086] Controlling the monocular camera to sequentially align with each of the plurality of framing blocks;

[0087] When the monocular camera is aimed at any of the viewfinders, controlling the monocular camera to zoom from the first zoom factor to a second zoom factor, so that a feature point in the aimed viewfinder moves;

[0088] Determining a first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor;

[0089] determining a moving speed of the feature point according to the first moving distance;

[0090] Determine the depth value of the aligned view block according to the moving speed of the feature point, the second zoom factor and a preset coefficient, so as to obtain the depth values ​​of the multiple view blocks of the monocular camera at the view angle;

[0091] When the monocular camera fails to capture the scene to be modeled in a panoramic manner, controlling the monocular camera to rotate so as to change the viewing angle of the monocular camera;

[0092] Executing the step of controlling the monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current framing angle, and dividing the framing image at the first zoom factor into a plurality of framing blocks;

[0093] In the case that the monocular camera has captured a panoramic view of the scene to be modeled, a three-dimensional model of the scene to be modeled is constructed according to the depth values ​​of the multiple viewing blocks at the multiple viewing angles obtained.

[0094] In some embodiments, when determining the depth value of the aligned viewfinder according to the moving speed of the feature point, the second zoom factor and a preset coefficient, the processor 101 is configured to implement:

[0095] Determining a depth value of the feature point according to a moving speed of the feature point, the second zoom factor, and a preset coefficient;

[0096] The depth value of the aligned framing block is determined according to the depth value of the feature point.

[0097] In some embodiments, when determining the depth value of the feature point according to the moving speed of the feature point, the second zoom factor and a preset coefficient, the processor 101 is used to implement:

[0098] The product of the moving speed of the feature point and the second zoom factor is determined, and the preset coefficient is divided by the product to obtain a depth value of the feature point.

[0099] In some embodiments, when determining the depth value of the aligned framing block according to the depth value of the feature point, the processor 101 is configured to implement:

[0100] In the case where there is only one feature point, taking the depth value of the feature point as the depth value of the aligned framing block;

[0101] In the case that there are multiple feature points, an average depth value is calculated according to the depth values ​​of the multiple feature points, and the average depth value is used as the depth value of the aligned framing block.

[0102] In some embodiments, when the processor 101 controls the monocular camera to zoom from the first zoom factor to the second zoom factor, it is configured to implement:

[0103] According to a preset zoom rate, the monocular camera is controlled to zoom from the first zoom factor to a second zoom factor.

[0104] In some embodiments, after determining the first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor, the processor 101 is further configured to:

[0105] Determining whether the first moving distance is greater than or equal to a preset distance threshold;

[0106] When the first moving distance is greater than or equal to a preset distance threshold, the step of determining the moving speed of the feature point according to the first moving distance is performed.

[0107] In some embodiments, after determining whether the first moving distance is greater than or equal to a preset distance threshold, the processor 101 is further configured to implement:

[0108] When the first moving distance is less than a preset distance threshold, controlling the monocular camera to zoom to a third zoom factor, wherein the third zoom factor is greater than the second zoom factor;

[0109] Determine a second moving distance of the feature point, where the second moving distance is a moving distance of the feature point after the monocular camera zooms from the first zoom factor to the third zoom factor or a moving distance of the feature point after the monocular camera zooms from the second zoom factor to the third zoom factor;

[0110] When the second moving distance is greater than or equal to a preset distance threshold, the moving speed of the feature point is determined according to the second moving distance, and the depth value of the aligned viewfinder is determined according to the moving speed of the feature point, the third zoom factor and a preset coefficient.

[0111] In some embodiments, the first zoom factor is smaller than the second zoom factor, or the first zoom factor is larger than the second zoom factor.

[0112] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned scene modeling method embodiment, and will not be repeated here.

[0113] An embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any scene modeling method provided in the description of the embodiment of the present invention.

[0114] The storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0115] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0116] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0117] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A scene modeling method based on a variable-focus monocular camera, characterized in that: include: Controlling a monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current viewing angle, and dividing a viewing image at the first zoom factor into a plurality of viewing blocks; Controlling the monocular camera to sequentially align with each of the plurality of framing blocks; When the monocular camera is aimed at any of the viewfinders, controlling the monocular camera to zoom from the first zoom factor to a second zoom factor, so that a feature point in the aimed viewfinder moves; Determining a first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor; determining a moving speed of the feature point according to the first moving distance; Determine the depth value of the aligned view block according to the moving speed of the feature point, the second zoom factor and a preset coefficient, so as to obtain the depth values ​​of the multiple view blocks of the monocular camera at the view angle; When the monocular camera fails to capture the scene to be modeled in a panoramic manner, controlling the monocular camera to rotate so as to change the viewing angle of the monocular camera; Executing the step of controlling the monocular camera located in the scene to be modeled to zoom to a first zoom factor at a current framing angle, and dividing the framing image at the first zoom factor into a plurality of framing blocks; In the case that the monocular camera has captured a panoramic view of the scene to be modeled, a three-dimensional model of the scene to be modeled is constructed according to the depth values ​​of the multiple viewing blocks at the multiple viewing angles obtained.

2. The scene modeling method according to claim 1, characterized in that: The step of determining the depth value of the aligned framing block according to the moving speed of the feature point, the second zoom factor and a preset coefficient comprises: Determining a depth value of the feature point according to a moving speed of the feature point, the second zoom factor, and a preset coefficient; The depth value of the aligned framing block is determined according to the depth value of the feature point.

3. The scene modeling method according to claim 2, characterized in that: The step of determining the depth value of the feature point according to the moving speed of the feature point, the second zoom factor and a preset coefficient includes: The product of the moving speed of the feature point and the second zoom factor is determined, and the preset coefficient is divided by the product to obtain a depth value of the feature point.

4. The scene modeling method according to claim 2, characterized in that: The step of determining the depth value of the aligned framing block according to the depth value of the feature point comprises: In the case where there is only one feature point, taking the depth value of the feature point as the depth value of the aligned framing block; In the case that there are multiple feature points, an average depth value is calculated according to the depth values ​​of the multiple feature points, and the average depth value is used as the depth value of the aligned framing block.

5. The scene modeling method according to claim 1, characterized in that: The controlling the monocular camera to zoom from the first zoom factor to the second zoom factor includes: According to a preset zoom rate, the monocular camera is controlled to zoom from the first zoom factor to a second zoom factor.

6. The scene modeling method according to any one of claims 1 to 5, characterized in that: After determining a first moving distance of the feature point after zooming from the first zoom factor to the second zoom factor, the method further includes: Determining whether the first moving distance is greater than or equal to a preset distance threshold; When the first moving distance is greater than or equal to a preset distance threshold, the step of determining the moving speed of the feature point according to the first moving distance is performed.

7. The scene modeling method according to claim 6, characterized in that: After determining whether the first moving distance is greater than or equal to a preset distance threshold, the method further includes: When the first moving distance is less than a preset distance threshold, controlling the monocular camera to zoom to a third zoom factor, wherein the third zoom factor is greater than the second zoom factor; Determine a second moving distance of the feature point, where the second moving distance is a moving distance of the feature point after the monocular camera zooms from the first zoom factor to the third zoom factor or a moving distance of the feature point after the monocular camera zooms from the second zoom factor to the third zoom factor; When the second moving distance is greater than or equal to a preset distance threshold, the moving speed of the feature point is determined according to the second moving distance, and the depth value of the aligned viewfinder is determined according to the moving speed of the feature point, the third zoom factor and a preset coefficient.

8. The scene modeling method according to any one of claims 1 to 5, characterized in that: The first zoom factor is smaller than the second zoom factor.

9. A computer device, characterized in that: The computer device includes a monocular camera with variable focus, a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the monocular camera, the processor and the memory, wherein when the computer program is executed by the processor, the scene modeling method according to any one of claims 1 to 8 is implemented.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores a computer program, and the computer program can be executed by one or more processors to implement the scene modeling method according to any one of claims 1 to 8.

Images