Scene construction method, device, equipment and medium based on cloud-edge collaboration
By acquiring cloud images on edge nodes, determining camera poses and building three-dimensional models using matching and construction algorithms, training scene generation models, solving the problem that high-precision multi-stylized scenes cannot be constructed in the existing technology, achieving more accurate and Diversified scenario generation reduces network pressure and data leakage risks.
Patent Information
- Application Number
- CN202411142500.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The open source system used in the prior art to manage containerized applications on multiple hosts lacks the ability to directly support scene reconstruction and stylized generation, resulting in the inability to effectively build high-precision, multi-stylized scenarios.
Using a cloud-edge collaboration method, by obtaining the images to be matched sent by the cloud, using a preset matching algorithm to obtain the feature matching rate, determining the target camera pose, and constructing a three-dimensional reconstruction model through a preset construction algorithm. Training the preset scene generation model to build the target scene image.
It realizes the generation of diverse and more accurate scene images on edge nodes, which reduces the pressure on cloud networks and data centers, reduces the risk of data leakage, and avoids the limitations of scene construction caused by open source systems.
Smart Images

Figure CN118657890B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of the Internet of Things, and in particular to a scene construction method, device, equipment and medium based on cloud-edge collaboration. Background Art
[0002] With the rapid development of science and technology, people are increasingly inclined to integrate the wide coverage of cloud computing with the instant response advantages of edge computing to drive diversified service innovation. Among them, scene construction, as one of the core technologies in the fields of virtual reality and game design, is of self-evident importance. Scene reconstruction and stylized generation are particularly critical, as they can give the virtual world a vivid and realistic texture and unlimited creative possibilities. However, the open source systems used to manage containerized applications on multiple hosts in the existing technology often lack the ability to directly support scene reconstruction and stylized generation, or because of their cumbersome deployment process and high management complexity, the accuracy and style diversity of the generated scenes are limited. As a result, it is impossible to effectively build high-precision, multi-stylized scenes in the existing technology. Summary of the invention
[0003] The embodiments of the present invention provide a scene construction method, device, equipment and medium based on cloud-edge collaboration, aiming to solve the problem that scenes cannot be effectively constructed in the prior art.
[0004] In the first aspect, an embodiment of the present invention provides a scene construction method based on cloud-edge collaboration, which is applied to an edge node and includes: obtaining images to be matched sent by the cloud, obtaining feature matching rates between the images to be matched through a preset matching algorithm, and determining a target camera pose according to the feature matching rate; constructing a three-dimensional reconstruction model using a preset construction algorithm for depth images and color images collected by the target camera pose; training a preset scene generation model based on the three-dimensional reconstruction model, and constructing a target scene image using the trained preset scene generation model for the images to be constructed.
[0005] In the second aspect, an embodiment of the present invention also provides a scene construction device based on cloud-edge collaboration, which is applied to an edge node and includes: a matching unit, used to obtain images to be matched sent by the cloud, obtain the feature matching rate between the images to be matched through a preset matching algorithm, and determine the target camera pose according to the feature matching rate; a construction unit, used to construct a three-dimensional reconstruction model of the depth image and the color image collected by the target camera pose through a preset construction algorithm; a generation unit, used to train a preset scene generation model according to the three-dimensional reconstruction model, and construct the target scene image through the trained preset scene generation model with the image to be constructed.
[0006] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.
[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions can implement the above method when executed by a processor.
[0008] The embodiment of the present invention provides a scene construction method, device, equipment and medium based on cloud-edge collaboration. The method is applied to an edge node, and includes: obtaining an image to be matched sent by the cloud, obtaining a feature matching rate between the images to be matched by a preset matching algorithm, and determining the target camera pose according to the feature matching rate; constructing a three-dimensional reconstruction model by using a preset construction algorithm for the depth image and the color image collected by the target camera pose; training a preset scene generation model according to the three-dimensional reconstruction model, and constructing a target scene image by using the trained preset scene generation model for the image to be constructed. The embodiment of the present invention receives the image to be matched at the edge node, processes the image to determine the target camera pose and obtains the three-dimensional reconstruction model, and trains the preset scene generation model according to the three-dimensional reconstruction model, wherein a variety of target scene images can be generated in the preset scene generation model. The generation of scene images by the edge node reduces the pressure on the cloud network and the data center, reduces the upload of user privacy data, effectively reduces the risk of data leakage, and avoids the limitations of scene construction caused by the open source system, so as to effectively construct a scene image with diversity and more accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0010] Figure 1 A schematic diagram of a process flow of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0011] Figure 2 A schematic diagram of a sub-process of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0012] Figure 3 A schematic diagram of a sub-process of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0013] Figure 4 A schematic diagram of a sub-process of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0014] Figure 5 A schematic diagram of a sub-process of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0015] Figure 6 A schematic diagram of a sub-process of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention;
[0016] Figure 7 A schematic block diagram of a scene construction device based on cloud-edge collaboration provided by an embodiment of the present invention;
[0017] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0020] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0021] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0022] See also Figure 1 , Figure 1A flow chart of a scene construction method based on cloud-edge collaboration provided in an embodiment of the present invention. The scene construction method based on cloud-edge collaboration in this embodiment can be applied to edge nodes in a cloud-edge collaboration architecture. An edge node controller module and a cloud routing module are deployed at the cloud, wherein the edge node controller module, as an extended Kubernetes (open source system) controller, is responsible for managing edge nodes and corresponding metadata to ensure that data is transmitted to the target edge node in a directional manner; the cloud routing module, as a Web Socket (a protocol for full-duplex communication on a single TCP (Transmission Control Protocol) connection) server, implements a communication bridge with the Kubernetes API (Application Programming Interface) server and edge node components, monitors cloud state changes, caches and pushes messages to the edge in real time, and receives feedback information from the edge. An edge container orchestration module, an edge routing module, an edge storage module, and a metadata management module are deployed at the edge node. The current scene can be generated into a corresponding three-dimensional scene image through the edge node to reduce the pressure on the cloud network and data center, while avoiding the limitations of scene construction caused by the open source system to effectively construct a scene image with diversity and more accuracy.
[0023] Figure 1 1 is a flow chart of a scene construction method based on cloud-edge collaboration provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps S110-S130.
[0024] S110, obtaining the to-be-matched images sent from the cloud, obtaining the feature matching rate between the to-be-matched images through a preset matching algorithm, and determining the target camera pose according to the feature matching rate.
[0025] In this embodiment, the image to be matched is a set of images of different angles of an indoor scene obtained by a pan-tilt image acquisition device, wherein the pan-tilt image refers to an image taken by a camera installed on the pan-tilt. The pan-tilt is a supporting device used to install and fix a mobile phone, camera or video camera, allowing them to rotate arbitrarily, so as to facilitate users to shoot or monitor from multiple angles. The feature matching rate is the matching degree of feature points between two frames of images to be matched, and the target camera posture is the position and angle of the final placement of the camera. The pan-tilt image acquisition device sends the collected image to be matched to the cloud, and the cloud sends it to the edge node. After receiving the image to be matched, the edge node obtains the feature matching rate between the images to be matched through a preset matching algorithm, wherein the preset matching algorithm is an algorithm for obtaining the feature matching rate between two frames of images, for example, the GMS algorithm, which matches by using SIFT (Scale-invariant feature transform) features, and marks correct and incorrect matches according to the true value, and then counts the number of matches in the small neighborhood where each matching feature point is located, so as to determine the feature matching rate. The target camera pose is determined according to the feature matching rate, for example, the pose corresponding to the image to be matched with the highest feature matching rate is used as the target camera pose. The target camera pose is determined by obtaining the feature matching rate between the images to be matched according to the preset matching algorithm, so as to optimize the camera pose, so as to obtain a more accurate captured image according to the optimized pose.
[0026] In one embodiment, if Figure 2 As shown, the step S110 includes steps S111-S112:
[0027] S111, obtaining feature point pairs between any two frames of the to-be-matched images by a preset feature acquisition method;
[0028] S112. Obtain the corresponding feature matching rate of the feature point pair through a preset matching algorithm, wherein the preset matching algorithm is a GMS algorithm.
[0029] In this embodiment, the feature point pair is the feature point corresponding to any two frames of images, and the feature point refers to a point with significance and uniqueness in the image. A set of feature point pairs between any two frames of the images to be matched is obtained. Specifically, the key points in the image can be found in the first frame image and the second frame image by edge detection, corner detection and other methods to identify the feature points, and the feature points in the first frame image and the second frame image are matched. Euclidean distance or correlation coefficient can be used to determine whether they match. If they match, they are determined as feature point pairs, wherein the specific method of obtaining feature points and matching feature points is not limited. The feature point pairs are passed through the GMS algorithm to obtain the corresponding feature matching rate, and the GMS algorithm determines the feature matching rate by counting the number of matches in the neighborhood where each matched feature is located. The feature matching rate between the feature point pairs is obtained according to the GMS algorithm to understand the similarity between the images to be matched, so as to determine the target camera pose according to the most matched image.
[0030] In one embodiment, if Figure 3 As shown, the step S110 includes steps S113-S114.
[0031] S113, comparing the feature matching rates between the images to be matched, and determining the highest feature matching rate;
[0032] S114. Determine the target camera pose according to the camera pose of the to-be-matched image corresponding to the highest feature matching rate.
[0033] In this embodiment, the edge node compares all the feature matching rates calculated between the images to be matched to determine the highest feature matching rate, and optimizes the camera pose according to the highest feature matching rate. Specifically, the camera position and angle of the image corresponding to the highest feature matching rate are determined as the target camera pose. It can be understood that because the feature points between the images are very matched, the camera pose corresponding to any one of the two frames of images corresponding to the highest feature matching rate can be selected as the target camera pose. The target camera pose is determined to facilitate the acquisition of an accurate basic image of the scene image.
[0034] S120, constructing a three-dimensional reconstruction model using a preset construction algorithm using the depth image and the color image collected by the target camera posture.
[0035] In this embodiment, the depth image is an image acquired by a depth camera in a pan-tilt image acquisition device, and the depth image refers to an image with the distance (depth) from the image acquisition device to each point in the scene as a pixel value, which directly reflects the geometric shape of the visible surface of the scene. Specifically, the depth image can be acquired by an Xtion camera in a pan-tilt image acquisition device, and the Xtion camera is a depth camera that supports simultaneous capture of color images and depth image outputs. The Xtion camera acquires the depth image and the color image through the target camera posture, and sends the depth image and the color image to the edge node through the cloud, and the edge node constructs a three-dimensional reconstruction model through a preset construction algorithm, wherein the three-dimensional reconstruction model is a model in which all the coordinates of the depth image are modified to the three-dimensional point cloud coordinates in the global coordinate system, that is, a model after the object in the image is turned into a three-dimensional model. The preset construction algorithm is an algorithm used for three-dimensional surface reconstruction in computer graphics and computer vision, for example, the TSDF (Truncated Signed Distance Function) algorithm, the TSDF algorithm can calculate the signed distance from each voxel in the scene to the surface of the object, and use this information to reconstruct the three-dimensional surface model of the object. A three-dimensional reconstruction model is obtained by using the depth image and color image collected according to the target camera posture and a preset construction algorithm to accurately adjust the current collected image into a 3D model, so as to avoid the problem of inaccurate constructed scenes due to inaccurate generated three-dimensional reconstruction models.
[0036] In one embodiment, if Figure 4 As shown, the step S120 includes steps S121-S122.
[0037] S121, performing point cloud coordinate conversion on pixel coordinates and depth coordinates by using a preset three-dimensional point cloud formula to determine point cloud coordinates, wherein a coordinate system of the point cloud coordinates is a preset global coordinate system, the color image includes the pixel coordinates, and the depth image includes the depth coordinates;
[0038] S122: Determine the three-dimensional reconstructed model according to the point cloud coordinates.
[0039] In this embodiment, the color image is a planar 2D image, and the image has pixel coordinates, which are expressed as (u, v), where u is the horizontal coordinate and v is the vertical coordinate. The origin of the coordinate system can be the lower left corner of the image, which is not limited. The depth image includes the depth coordinate, which can be expressed as (x, y, d), which represent the horizontal coordinate, the vertical coordinate and the depth value respectively. The preset three-dimensional point cloud formula is: Where s is the ratio coefficient between the actual depth and the measured depth. The actual depth refers to the real physical distance from a point in the scene to the camera (or sensor). This distance is the actual spatial distance between the object and the camera and is not limited by the measurement device or method. The measured depth is the estimated distance from each point in the scene to the camera obtained through the depth image. f u and f v is the focal length of the camera, u c with u v is the optical center parameter of the camera. It can be understood that the parameters of the focal length and the optical center are known parameters. x', y', z' are the horizontal coordinates, vertical coordinates and depth values of the point cloud coordinates. The point cloud coordinates can be obtained through the pixel coordinates and the depth value in the depth coordinates. The point cloud coordinates are the coordinates converted from the depth coordinates to a preset global coordinate system, wherein the preset global coordinate system is the global coordinate system used for observation and measurement through multiple sites. And the three-dimensional reconstruction model is determined according to the point cloud coordinates, that is, the image composed of the point cloud coordinates in the preset global coordinate system is the three-dimensional reconstruction model. The pixel coordinates and the depth coordinates are converted into the point cloud coordinates through a preset three-dimensional point cloud formula to determine the point cloud coordinates so as to convert the point clouds of each single-site coordinate system of the gimbal to the multi-site global coordinate system of the gimbal, and obtain a general three-dimensional reconstruction model.
[0040] In one embodiment, if Figure 5 As shown, step S120 includes steps S1201-S1202.
[0041] S1201, if the posture of the current depth image and / or the scene in the image changes, fusing the distance value of the depth image of the previous frame with the distance value of the current depth image through a preset fusion algorithm;
[0042] S1202: Determine a new depth image according to the fused distance values, and determine a new 3D reconstruction model according to the new depth image.
[0043] In this embodiment, the distance value is a signed distance representing the voxel to the surface of the nearest object, wherein the voxel is the abbreviation of volume element, which is the smallest unit of digital data in the three-dimensional space segmentation. It is similar to a pixel in a two-dimensional space. The signed distance is a distance value represented by a signed distance function. The sign (positive or negative) of the distance can be used to indicate the positional relationship of the point relative to the object or surface. When the posture of the camera that acquires the depth image or the scene in the depth image changes, the distance value of the depth image of the previous frame can be fused with the distance value of the current depth image through a preset fusion algorithm. The specific fusion algorithm is: in, is the distance value after fusion, and are the distance value and weight value of the previous depth image, respectively. and They are the distance value and weight value of the current depth image, respectively, wherein the weight value can be obtained through information such as the angle between the projection light and the surface normal vector, and there is no limitation on this. The fused distance value can reflect the latest position and shape information of the object in the scene relative to the camera, so as to determine the new depth image, and determine the new 3D reconstruction model through the preset construction algorithm based on the new depth image. By obtaining the fused distance value to determine the new 3D reconstruction model corresponding to the new depth image, the preset construction algorithm used can be continuously updated, thereby generating a more complete and accurate depth map.
[0044] S130, training a preset scene generation model according to the three-dimensional reconstruction model, and constructing a target scene image by passing the image to be constructed through the trained preset scene generation model.
[0045] In this embodiment, the target scene image is a simulated stereoscopic scene image. Specifically, a simulated and 3D living room scene image can be constructed based on the image of the living room, and the living room scene image is the target scene image. The preset scene generation model is an AIGC (Artificial Intelligence Generated Content, artificial intelligence generation) deep generation neural model network based on Stable Diffusion (image generation model) technology, wherein the AIGC deep generation neural model network can use artificial intelligence technology and algorithms to automatically generate various forms of content. Specifically, it can autonomously generate and create new target scene images based on training data and generation algorithm models. Therefore, it is necessary to use the generated three-dimensional reconstruction model to train the AIGC deep generation neural model network so that it can generate the target scene image according to the trained reconstruction model, and the AIGC deep generation neural model network is constructed based on the Stable Diffusion, then it can compress the image and add noise to generate the target scene image. Among them, the process of training the preset scene generation model through the three-dimensional reconstruction model is not limited, and it can enable the AIGC deep generation neural model network to automatically generate scene images. The image to be constructed is a depth image collected by the pan-tilt image acquisition device and needs to be used for scene construction. The depth image collected by the camera is input into the trained preset scene generation model, and the preset scene generation model can construct and output the target scene image. By passing the image to be constructed through the trained preset scene generation model to construct the target scene image, the scene construction can be performed with higher accuracy and efficiency.
[0046] In one embodiment, if Figure 6 As shown, the step S130 includes steps S131-S132.
[0047] S131, controlling the image generation model to construct a scene according to the image to be constructed according to preset conditions through the condition control model;
[0048] S132. When the image generation model performs scene construction, the internal parameters of the image generation model are adjusted by the fine-tuning control model to construct the target scene image.
[0049] In this embodiment, the preset scene generation model includes a fine-tuning control model, an image generation model and a conditional control model. The fine-tuning control model is a LoRA model, and the LoRA model can improve the performance of the model through a small amount of parameter adjustment. The image generation model is a Stable Diffusion (SD) image generation model, which combines the diffusion process in physics, the variational autoencoder in the field of machine learning, and a reversible network to achieve high-quality image generation. The conditional control model is a ControlNet model, which can control the Stable Diffusion image generation model to support additional input conditions, such as edge maps, segmentation maps, etc. The image generation model is controlled by the conditional control model according to preset conditions to construct a scene image. Specifically, the Stable Diffusion image generation model can compress the image to be constructed through a variational autoencoder, and then perform noise prediction and image reconstruction to remove noise in the image to generate a clear image. When the Stable Diffusion image generation model generates an image, the ControlNet model controls the Stable Diffusion image generation model according to additional preset conditions to generate a scene image that meets the conditions. The preset conditions are additional input conditions, such as images with personalized edge maps, segmentation maps, etc., to assist the image generation model in building an accurate scene image based on the image to be built. While the image generation model generates the scene image, the LoRA model adjusts the internal parameters of the image generation model to fine-tune the target scene image, so as to adjust the weights of certain processing layers in the Stable Diffusion model to enable it to better generate scene images. There is no limitation on which internal parameters are adjusted, and the LoRA model can operate according to the specific image to be built. A variety of scene images can be built through the fine-tuning control model, image generation model and conditional control model in the preset scene generation model.
[0050] In one embodiment, step S130 includes step S1301 after step S130.
[0051] S1301. Send the target scene image to the cloud through a preset communication method, wherein the cloud can view the target scene image.
[0052] In this embodiment, the preset communication mode is a mode of communication between the cloud and the communication node. In this embodiment, the Web Socket communication protocol can be used as the preset communication mode, wherein the Web Socket communication protocol is a protocol for full-duplex communication on a single TCP connection, which can make data exchange between the client and the server easier. The edge node sends the generated target scene image to the cloud, so that the user can view the target scene image at the cloud. Data synchronization between the cloud and the communication node can be achieved by sending the target scene image to the cloud.
[0053] Figure 7 is a schematic block diagram of a scene construction device 200 based on cloud-edge collaboration provided by an embodiment of the present invention. Figure 7 As shown, corresponding to the above-mentioned scene construction method based on cloud-edge collaboration, the present invention also provides a scene construction device based on cloud-edge collaboration. The scene construction device based on cloud-edge collaboration includes a unit for executing the above-mentioned scene construction method based on cloud-edge collaboration, and the device can be configured in a desktop computer, a tablet computer, a laptop, and other terminals. For details, please refer to Figure 7 The scene construction device based on cloud-edge collaboration includes a matching unit 210, a construction unit 220 and a generation unit 230.
[0054] The matching unit 210 is used to obtain the to-be-matched images sent by the cloud, obtain the feature matching rate between the to-be-matched images through a preset matching algorithm, and determine the target camera pose according to the feature matching rate.
[0055] In one embodiment, the matching unit 210 includes a point pair unit and a calculation unit.
[0056] A point pair unit, used for acquiring feature point pairs between any two frames of the to-be-matched images by a preset feature acquisition method;
[0057] The calculation unit is used to obtain the corresponding feature matching rate of the feature point pair through a preset matching algorithm, wherein the preset matching algorithm is a GMS algorithm.
[0058] In one embodiment, the matching unit 210 includes a comparing unit and a first determining unit.
[0059] A comparison unit, used for comparing the feature matching rates between the images to be matched to determine the highest feature matching rate;
[0060] The first determining unit is used to determine the target camera pose according to the camera pose of the to-be-matched image corresponding to the highest feature matching rate.
[0061] The construction unit 220 is used to construct a three-dimensional reconstruction model using a preset construction algorithm using the depth image and the color image collected by the target camera posture.
[0062] In one embodiment, the construction unit 220 includes a conversion unit and a second determination unit.
[0063] A conversion unit, configured to perform point cloud coordinate conversion on pixel coordinates and depth coordinates by using a preset three-dimensional point cloud formula to determine point cloud coordinates, wherein a coordinate system of the point cloud coordinates is a preset global coordinate system, the color image includes the pixel coordinates, and the depth image includes the depth coordinates;
[0064] The second determining unit is used to determine the three-dimensional reconstructed model according to the point cloud coordinates.
[0065] In one embodiment, the construction unit 220 includes a fusion unit and a third determination unit.
[0066] A fusion unit, configured to fuse the distance value of the depth image of the previous frame with the distance value of the current depth image by using a preset fusion algorithm if the posture of the current depth image and / or the scene in the image changes;
[0067] The third determining unit is used to determine a new depth image according to the fused distance values, and determine a new 3D reconstruction model according to the new depth image.
[0068] The generating unit 230 is used to train a preset scene generating model according to the 3D reconstruction model, and construct a target scene image by passing the image to be constructed through the trained preset scene generating model.
[0069] In one embodiment, the generating unit 230 includes a noise adding unit and a constructing sub-unit.
[0070] A noise adding unit, configured to control the image generation model to construct a scene according to the image to be constructed through the condition control model according to preset conditions;
[0071] A construction subunit is used to adjust the internal parameters of the image generation model to construct the target scene image when the image generation model performs scene construction through the fine-tuning control model.
[0072] In one embodiment, the generating unit 230 includes a sending unit.
[0073] A sending unit is used to send the target scene image to the cloud through a preset communication method, wherein the cloud can view the target scene image.
[0074] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned scene construction device 200 based on cloud-edge collaboration and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.
[0075] The above-mentioned scene construction device based on cloud-edge collaboration can be implemented in the form of a computer program. The computer program can be used in Figure 8 Runs on the computer device shown.
[0076] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal or a server, wherein the terminal may be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of multiple servers.
[0077] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .
[0078] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, may enable the processor 502 to execute a scene construction method based on cloud-edge collaboration.
[0079] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .
[0080] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a scene construction method based on cloud-edge collaboration.
[0081] The network interface 505 is used to communicate with other devices over the network. Figure 8The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0082] The processor 502 is used to run a computer program 5032 stored in the memory to implement the steps of the above method.
[0083] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0084] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.
[0085] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the above method.
[0086] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.
[0087] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0088] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0089] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0091] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A scene construction method based on cloud-edge collaboration, It is characterized in that The method is applied to an edge node, and the method comprises: Obtain the to-be-matched images sent by the cloud, obtain the feature matching rate between the to-be-matched images through a preset matching algorithm, and determine the target camera pose according to the feature matching rate; The depth image and the color image acquired by the target camera posture are used to construct a three-dimensional reconstruction model through a preset construction algorithm; Training a preset scene generation model according to the three-dimensional reconstruction model, and passing the image to be constructed through the trained preset scene generation model to construct a target scene image; The step of obtaining the feature matching rate between the images to be matched by using a preset matching algorithm includes: Acquire feature point pairs between any two frames of the to-be-matched images by a preset feature acquisition method; Obtaining the corresponding feature matching rate of the feature point pair through a preset matching algorithm, wherein the preset matching algorithm is a GMS algorithm; The step of determining the target camera pose according to the feature matching rate comprises: Comparing the feature matching rates between the images to be matched to determine the highest feature matching rate; Determine the target camera pose according to the camera pose of the to-be-matched image corresponding to the highest feature matching rate; The step of constructing a three-dimensional reconstruction model using a preset construction algorithm using a depth image and a color image includes: The pixel coordinates and the depth coordinates are converted into point cloud coordinates by a preset three-dimensional point cloud formula to determine the point cloud coordinates, wherein the coordinate system of the point cloud coordinates is a preset global coordinate system, the color image includes the pixel coordinates, and the depth image includes the depth coordinates; Determine the three-dimensional reconstructed model according to the point cloud coordinates, wherein an image formed by the point cloud coordinates in the preset global coordinate system is the three-dimensional reconstructed model; After the step of constructing a three-dimensional reconstruction model using the depth image and the color image through a preset construction algorithm, the method further includes: If the posture of the current depth image and / or the scene in the image changes, the distance value of the depth image of the previous frame is fused with the distance value of the current depth image through a preset fusion algorithm; A new depth image is determined according to the fused distance values, and a new three-dimensional reconstruction model is determined according to the new depth image.
2. The method according to claim 1, It is characterized in that The preset scene generation model includes a fine-tuning control model, an image generation model and a conditional control model. The step of constructing a target scene image by passing the trained preset scene generation model through the image to be constructed includes: Controlling the image generation model to construct a scene according to the image to be constructed by the conditional control model according to preset conditions; The fine-tuning control model is used to adjust the internal parameters of the image generation model when the image generation model performs scene construction to construct the target scene image.
3. The method according to claim 2, It is characterized in that After the step of constructing the target scene image by using the trained preset scene generation model for the image to be constructed, the method further includes: The target scene image is sent to the cloud via a preset communication method, wherein the target scene image can be viewed in the cloud.
4. A scene construction device based on cloud-edge collaboration, It is characterized in that The device is applied to an edge node, and comprises a unit for executing the method according to any one of claims 1 to 3.
5. A computer device, It is characterized in that The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 3 when executing the computer program.
6. A storage medium, It is characterized in that The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 3 can be implemented.
