Method, device, equipment and medium for determining the position of ball nodes in space grid

Abstract

The embodiment of the present application discloses a method, device, equipment and medium for determining the position of a ball node in a space grid. The method includes: obtaining a target video of a target space grid; obtaining a target point cloud according to the target video; dividing the space where the target point cloud is located into multiple first grids according to the grid overlap rate, and the first grid is a cube with a side length of the maximum ball node diameter; extracting the grid point cloud located in the first grid; performing sphere fitting processing on the grid point cloud to obtain multiple candidate sphere point clouds, and determining the target sphere point cloud from the multiple candidate sphere point clouds; determining at least one candidate sphere set based on the fitting error threshold, the fitting error value of each target sphere point cloud and the preset minimum distance between spheres; determining the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; according to the current node position of each ball node in the target sphere set. The position of the ball node in the space grid can be obtained by implementing the method of the embodiment of the present application.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and medium for determining the position of a ball node in a space grid. Background Art

[0002] The space grid structure is a mesh space bar structure composed of many bars through ball nodes according to a certain rule. Since its advent, the space grid has been welcomed by people for its efficient force-bearing performance, novel and beautiful form, and fast and convenient construction. Space grids can be seen in stadiums, exhibition centers, cultural facilities, transportation hubs and even industrial plants that require large spans and large spaces.

[0003] To ensure the safety of the space grid, after the space grid is installed, it is necessary to periodically analyze the stress state of the space grid. The stress state of the grid can be analyzed through the displacement of the ball nodes in the space grid. However, due to the large volume of the space grid, it is difficult to obtain the position of the ball nodes in the space grid using existing technology. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and medium for determining the position of a ball node in a space grid, which can obtain the position of the ball node in the space grid.

[0005] In a first aspect, an embodiment of the present application provides a method for determining the position of a ball node in a space grid, which comprises:

[0006] Acquire a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles;

[0007] Performing three-dimensional reconstruction processing on the target video to obtain a target point cloud;

[0008] Dividing the space where the target point cloud is located into a plurality of first grids according to a preset grid overlap rate, wherein the first grid is a cube with a side length equal to the diameter of the maximum spherical node;

[0009] For each first grid, extracting a grid point cloud located in the first grid from the target point cloud;

[0010] For each first grid, a sphere fitting process is performed on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and a sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds is determined as the target sphere point cloud of the first grid;

[0011] Determine at least one candidate sphere set from the plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres;

[0012] Determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set;

[0013] The current node position of each ball node in the target space grid is determined according to the target sphere set.

[0014] In a second aspect, the embodiment of the present application further provides a device for determining the position of a ball node in a space grid, which includes:

[0015] A transceiver unit, used to obtain a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles;

[0016] A processing unit is used to perform three-dimensional reconstruction processing on the target video to obtain a target point cloud; divide the space where the target point cloud is located into multiple first grids according to a preset grid overlap rate, and the first grid is a cube with a side length of the maximum ball node diameter; for each first grid, extract the grid point cloud located in the first grid from the target point cloud; for each first grid, perform sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determine the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid; based on a preset fitting error threshold, the fitting error value of each target sphere point cloud and a preset minimum distance between spheres, determine at least one candidate sphere set from the multiple target sphere point clouds; determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; and determine the current node position of each sphere node in the target space grid according to the target sphere set.

[0017] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and wherein the program instructions can implement the above method when executed by a processor.

[0019] The embodiments of the present application provide a method, device, equipment and medium for determining the position of a ball node in a space grid. The method includes: obtaining a target video, which is a video obtained by shooting a target space grid at multiple angles; performing three-dimensional reconstruction processing on the target video to obtain a target point cloud; dividing the space where the target point cloud is located into multiple first grids according to a preset grid overlap rate, wherein the first grid is a cube with a side length of a maximum ball node diameter; for each first grid, extracting a grid point cloud located in the first grid from the target point cloud; for each first grid, performing sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determining the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid; based on a preset fitting error threshold, the fitting error values ​​of each target sphere point cloud and a preset minimum distance between spheres, determining at least one candidate sphere set from the multiple target sphere point clouds; determining the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; and determining the current node position of each sphere node in the target space grid according to the target sphere set. The embodiment of the present application obtains a point cloud of the spatial grid by performing three-dimensional reconstruction of the spatial grid through a video of the spatial grid, and then obtains a spherical point cloud of each spherical node in the spatial grid based on the point cloud of the spatial grid, and determines the node position of each spherical node based on the spherical point cloud of each spherical node. It can be seen that the position of the spherical nodes in the spatial grid can be obtained through this solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of an application scenario of a method for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of a method for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a sub-process of a method for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of another sub-process of the method for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0025] Figure 5A schematic diagram of another sub-process of the method for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0026] Figure 6 A schematic block diagram of a device for determining the position of a ball node in a space grid provided in an embodiment of the present application;

[0027] Figure 7 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0030] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] The embodiments of the present application provide a method, device, equipment and medium for determining the position of a ball node in a space grid.

[0033] The executor of the method for determining the position of the ball nodes in the space grid may be the device for determining the position of the ball nodes in the space grid provided in the embodiment of the present application, or a computer device integrated with the device for determining the position of the ball nodes in the space grid, wherein the device for determining the position of the ball nodes in the space grid may be implemented in hardware or software, the computer device may be a terminal or a server, and the terminal may be a smart phone, a tablet computer, a PDA, or a laptop computer, etc.

[0034] See also Figure 1, Figure 1 Schematic diagram of an application scenario of the method for determining the position of a ball node in a space grid provided in an embodiment of the present application. The method for determining the position of a ball node in a space grid is applied to Figure 1 In the computer device in the invention, the computer device obtains a target video, which is a video obtained by shooting a target space grid frame from multiple angles; performs three-dimensional reconstruction processing on the target video to obtain a target point cloud; divides the space where the target point cloud is located into multiple first grids according to a preset grid overlap rate, and the first grid is a cube with a side length of the maximum ball node diameter; for each first grid, extracts a grid point cloud located in the first grid from the target point cloud; for each first grid, performs sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determines the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid; based on a preset fitting error threshold, the fitting error value of each target sphere point cloud and a preset minimum distance between spheres, determines at least one candidate sphere set from the multiple target sphere point clouds; determines the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; and determines the current node position of each sphere node in the target space grid frame according to the target sphere set.

[0035] Figure 2 FIG. 1 is a flow chart of a method for determining the position of a ball node in a space grid provided by an embodiment of the present application. Figure 2 As shown, the method includes the following steps S110-S180.

[0036] S110, obtaining a target video, wherein the target video is a video obtained by shooting a target space grid at multiple angles.

[0037] In this embodiment, the target space grid is a space grid that has been put into use and currently needs to obtain the position of the ball node; the target video can be acquired by the computer device itself (in this case, the computer device can be a drone with powerful computing capabilities, or an image acquisition device with powerful computing capabilities carried by a drone or a robotic arm), or it can be acquired by other image acquisition devices (drones, or image acquisition devices carried by drones or robotic arms) and then transmitted to the computer device. This embodiment does not limit the specific method of acquiring the target video.

[0038] The target video is a video obtained by shooting the target space grid from multiple angles, ensuring that the three-dimensional reconstruction of the space grid can be performed using the video.

[0039] S120: Perform three-dimensional reconstruction processing on the target video to obtain a target point cloud.

[0040] Specifically, see Figure 3 In some embodiments, step S120 includes:

[0041] S1201. Extract multiple key frame images from the target video according to a preset frame extraction rule.

[0042] Among them, the frame extraction rule can be to extract one frame every preset time interval, and the preset time interval can be 3 seconds. The specific preset time interval can also be adapted and adjusted according to actual conditions. This embodiment does not limit the specific time of the preset time interval.

[0043] For example, the total duration of the target video is 300 seconds, and a key frame image is extracted every 3 seconds, so 100 key frame images are obtained.

[0044] Furthermore, in order to improve the image quality, after obtaining the key frame image, each key frame needs to be denoised and enhanced.

[0045] S1202 extracts image feature points of each of the key frame images respectively, and performs feature point matching processing on a plurality of the image feature points to establish a feature point correspondence relationship between the image feature points.

[0046] In this embodiment, the image feature points of each of the key frame images may be extracted respectively according to a preset feature point extraction algorithm, and feature point matching and erroneous matching may be performed on the plurality of image feature points.

[0047] Among them, the preset feature point extraction algorithm can be a scale-invariant feature transform (SIFT) algorithm, a speeded up robust features (SURF) algorithm or an oriented fast and rotated brief (ORB) algorithm.

[0048] S1203: Based on the feature point correspondence, a camera motion trajectory is determined using a basic matrix and an essential matrix, and a multi-view geometric relationship of multiple frames of the key frame images is determined according to the camera motion trajectory.

[0049] In this embodiment, according to the correspondence relationship between the feature points, the camera motion trajectory is restored using the basic matrix and the essential matrix, wherein the camera motion trajectory is the motion trajectory when the image acquisition device acquires the target video.

[0050] The camera motion trajectory is composed of multiple postures, and different postures correspond to different key frame images. Therefore, this embodiment can determine the multi-view geometric relationship of multiple frames of the key frame images according to the camera motion trajectory.

[0051] S1204: Convert the image feature points of each of the key frame images into an initial point cloud based on the feature point correspondence relationship and the multi-view geometric relationship.

[0052] This embodiment is a step of sparsely reconstructing the point cloud of the target space grid. Specifically, based on the feature point correspondence and the multi-view geometric relationship, the three-dimensional coordinates of the image feature points of each key frame image are obtained by triangulation, and then a sparse point cloud is constructed based on the three-dimensional coordinates of each image feature point, and bundle adjustment (BA) optimization is performed to obtain an initial point cloud.

[0053] S1205: Perform dense reconstruction processing on the initial point cloud to obtain the target point cloud.

[0054] Specifically, this embodiment may use a dense reconstruction (Multi-View Stereo, MVS) algorithm to perform dense reconstruction processing on the initial point cloud to obtain a more accurate target point cloud.

[0055] In some embodiments, in order to improve the three-dimensional image display effect of the target space grid, after obtaining the target point cloud, multiple second grids are constructed for the target point cloud according to a preset surface reconstruction algorithm; texture mapping processing is performed on the multiple second grids based on multiple frames of the key frame images to obtain a space grid model.

[0056] The surface reconstruction algorithm may specifically be a Poisson reconstruction algorithm.

[0057] S130. Divide the space where the target point cloud is located into a plurality of first grids according to a preset grid overlap rate, where the first grid is a cube with a side length equal to the diameter of a maximum spherical node.

[0058] In this embodiment, the preset grid overlap rate can be 50%. The grid overlap rate can also be set to other values ​​according to actual needs, such as 30%. This application does not limit the specific value of the grid overlap rate.

[0059] Specifically, this embodiment divides the entire point cloud space where the target point cloud is located into a plurality of overlapping grids, and the size of the grid is set to the maximum ball node diameter.

[0060] Among them, the maximum ball node diameter is the maximum ball node set for the ball node in the industry. For example, in the current space grid, the ball node diameters are all in the interval [a, b]. At this time, the maximum ball node diameter is b.

[0061] In addition, in this embodiment, the first grids overlap with each other, which can avoid missing spheres at the boundaries.

[0062] S140 . For each first grid, extract a grid point cloud located in the first grid from the target point cloud.

[0063] In this embodiment, in each first grid, the point cloud in the grid is extracted as the grid point cloud of the corresponding grid.

[0064] In some embodiments, in order to reduce the amount of subsequent fitting calculations, before performing sphere fitting processing on the grid point cloud, it is also necessary to filter out useless first grids, specifically, first grids whose number of points in the grid point cloud is less than a preset point number threshold are filtered out. The specific value of the preset point number threshold can be adjusted according to actual needs, and the specific value is not limited in this example.

[0065] Subsequently, only the first grid remaining after filtering needs to be subjected to sphere fitting, which greatly reduces the amount of fitting calculations.

[0066] S150. For each first grid, perform sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determine the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid.

[0067] In this embodiment, in each first grid, sphere fitting processing is performed on the grid point cloud according to multiple candidate radii, and multiple candidate sphere point clouds are obtained.

[0068] Among them, multiple candidate radii are ball node radii commonly used in the industry.

[0069] In some embodiments, in order to speed up the fitting speed and reduce the amount of fitting calculation, for each first grid, this embodiment will also select a portion of multiple target fitting radii that match it from multiple candidate radii. For details, please refer to Figure 4 At this time, the step of performing sphere fitting processing on the grid point cloud according to multiple candidate radii in step S150 to obtain multiple candidate sphere point clouds includes:

[0070] S1501. Taking the average distance between each two points in the grid point cloud as an initial fitting parameter.

[0071] In this embodiment, in each first grid, the point distances between all the points in the grid are first calculated, and then the average value of all the point distances in the grid is calculated as the initial fitting parameter.

[0072] S1502: Divide the initial fitting parameter by a preset adjustment parameter to obtain a minimum fitting radius, and multiply the initial fitting parameter by the adjustment parameter to obtain a maximum fitting radius.

[0073] In this embodiment, the adjustment parameter is a value greater than 1, such as 1.5. This embodiment does not limit the specific value of the adjustment parameter.

[0074] S1503: Determine a radius among the multiple candidate radii that is larger than the minimum fitting radius and smaller than the maximum fitting radius as a target fitting radius.

[0075] In this embodiment, after the minimum fitting radius and the maximum fitting radius are calculated, the candidate radius within the interval between the minimum fitting radius and the maximum fitting radius is used as the target fitting radius.

[0076] S1504: Perform sphere fitting processing on the grid point cloud according to the target fitting radius to obtain multiple candidate sphere point clouds.

[0077] Specifically, this embodiment can perform sphere fitting processing on the grid point cloud based on the least squares method.

[0078] For example, there are 10 candidate radii and 5 target fitting radii that meet the conditions. At this time, it is only necessary to perform sphere fitting based on these 5 target fitting radii, thereby speeding up the fitting speed and reducing the amount of fitting calculation.

[0079] S160: Determine at least one candidate sphere set from the plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each of the target sphere point clouds, and a preset minimum distance between spheres.

[0080] Specifically, see Figure 5 In some embodiments, step S160 includes:

[0081] S1601, filtering target sphere point clouds whose fitting error values ​​are greater than the fitting error threshold among the plurality of target sphere point clouds.

[0082] In this embodiment, after obtaining a plurality of target sphere point clouds, firstly, based on a fitting error threshold, target sphere point clouds with larger fitting error values ​​are removed.

[0083] S1602: Determine the sphere distance between each target sphere point cloud obtained after filtering.

[0084] In this embodiment, the sphere distances between each pair of target sphere point clouds are determined according to the closest points between each target sphere point cloud.

[0085] S1603: Determine at least one candidate sphere set from the multiple target sphere point clouds obtained after filtering according to the sphere distance and the minimum distance between spheres.

[0086] In this embodiment, since the distance between each spherical node in the target space grid cannot be less than the specified minimum distance between spheres. If there is an abnormal distance between the calculated target sphere point clouds that is less than the minimum distance between spheres, it is necessary to remove one of the sphere point clouds corresponding to the abnormal distance based on the exhaustive algorithm. Since the removed sphere point clouds are different, the number of target sphere point clouds remaining in the end is also different. At this time, multiple candidate sphere sets may be obtained in the end.

[0087] S170: Determine the candidate sphere set that contains the largest number of target sphere point clouds as the target sphere set.

[0088] In this embodiment, the candidate sphere set with the largest number of target sphere point clouds is determined as the target sphere set.

[0089] In some embodiments, if there is more than one candidate sphere set with the largest number of target sphere point clouds, at this time, the total fitting error value of the sphere point clouds in each candidate sphere set with the largest number of point clouds can be further determined, and then the candidate sphere set with the smallest total fitting error value can be determined as the target sphere set.

[0090] S180. Determine the current node position of each spherical node in the target space grid according to the target spherical body set.

[0091] In this embodiment, the target sphere set includes a plurality of target sphere point clouds, each target sphere point cloud corresponds to a ball node, and the current node position of the corresponding ball node can be determined according to the coordinates of the corresponding target sphere point cloud.

[0092] In some embodiments, after obtaining the current node position of each ball node, the present embodiment will also automatically perform displacement analysis on each ball node. At this time, after determining the current node position of each ball node in the target space grid according to the target sphere set, the method further includes:

[0093] The ball node displacement analysis is performed according to the current ball node position of each ball node and the preset original node position of each ball node to obtain the ball node displacement analysis result.

[0094] Furthermore, if there is a ball node with a displacement greater than a displacement threshold in the ball node displacement analysis results, a space grid abnormality warning is automatically generated.

[0095] In order to enable the user to clearly view each ball node, after obtaining the target sphere set, this embodiment also uses a drawing library (such as matplotlib) to draw the target point cloud and the sphere point cloud in the target sphere set, and displays the center, spherical surface and related point cloud of each ball node in the target point cloud.

[0096] In summary, the embodiment of the present application obtains a point cloud of the spatial grid by performing three-dimensional reconstruction of the spatial grid through a video of the spatial grid, and then obtains a spherical point cloud of each spherical node in the spatial grid based on the point cloud of the spatial grid, and determines the node position of each spherical node based on the spherical point cloud of each spherical node. It can be seen that the position of the spherical nodes in the spatial grid can be obtained through this solution.

[0097] Figure 6 6 is a schematic block diagram of a device 600 for determining the position of a ball node in a space grid provided in an embodiment of the present application. Figure 6 As shown, corresponding to the above method for determining the position of a ball node in a space grid, the present application also provides a device 600 for determining the position of a ball node in a space grid. The device 600 for determining the position of a ball node in a space grid includes a unit for executing the above method for determining the position of a ball node in a space grid, and the device 600 for determining the position of a ball node in a space grid can be configured in a terminal or a server. Figure 6 The device 600 for determining the position of a ball node in a space grid includes a transceiver unit 601 and a processing unit 602, wherein:

[0098] The transceiver unit 601 is used to obtain a target video, where the target video is a video obtained by shooting the target space grid at multiple angles;

[0099] The processing unit 602 is used to perform three-dimensional reconstruction processing on the target video to obtain a target point cloud; divide the space where the target point cloud is located into multiple first grids according to a preset grid overlap rate, and the first grid is a cube with a side length of the maximum ball node diameter; for each first grid, extract the grid point cloud located in the first grid from the target point cloud; for each first grid, perform sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determine the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid; based on a preset fitting error threshold, the fitting error value of each target sphere point cloud and a preset minimum distance between spheres, determine at least one candidate sphere set from the multiple target sphere point clouds; determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; and determine the current node position of each sphere node in the target space grid according to the target sphere set.

[0100] In some embodiments, when the processing unit 602 performs the step of performing sphere fitting processing on the grid point cloud according to the multiple candidate radii to obtain the multiple candidate sphere point clouds, it is specifically used to:

[0101] Taking the average distance between each two points in the grid point cloud as an initial fitting parameter;

[0102] The initial fitting parameter is divided by a preset adjustment parameter to obtain a minimum fitting radius, and the initial fitting parameter is multiplied by the adjustment parameter to obtain a maximum fitting radius, where the adjustment parameter is a value greater than 1;

[0103] Determine a radius, among the plurality of candidate radii, that is larger than the minimum fitting radius and smaller than the maximum fitting radius as a target fitting radius;

[0104] The grid point cloud is subjected to sphere fitting processing according to the target fitting radius to obtain a plurality of candidate sphere point clouds.

[0105] In some embodiments, when executing the step of determining at least one candidate sphere set from a plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres, the processing unit 602 is specifically configured to:

[0106] Performing filtering processing on the target sphere point clouds whose fitting error values ​​are greater than the fitting error threshold among the plurality of target sphere point clouds;

[0107] Determine the sphere distance between each target sphere point cloud obtained after filtering;

[0108] At least one candidate sphere set is determined from the plurality of target sphere point clouds obtained after filtering according to the sphere distance and the minimum distance between spheres.

[0109] In some embodiments, after performing the step of extracting the grid point cloud located in the first grid from the target point cloud and before performing sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, the processing unit 602 is further used to:

[0110] The first grid whose number of points in the grid point cloud is less than a preset point number threshold is filtered.

[0111] In some embodiments, after executing the step of determining the current node position of each ball node in the target space grid according to the target sphere set, the processing unit 602 is further configured to:

[0112] The ball node displacement analysis is performed according to the current ball node position of each ball node and the preset original node position of each ball node to obtain the ball node displacement analysis result.

[0113] In some embodiments, when the processing unit 602 performs the step of performing three-dimensional reconstruction processing on the target video to obtain a target point cloud, it is specifically used to:

[0114] Extracting multiple key frame images from the target video according to a preset frame extraction rule;

[0115] Extracting image feature points of each of the key frame images respectively, and performing feature point matching processing on a plurality of the image feature points to establish a feature point correspondence relationship between the image feature points;

[0116] Based on the correspondence relationship of the feature points, a camera motion trajectory is determined using a basic matrix and an essential matrix, and a multi-view geometric relationship of multiple frames of the key frame images is determined according to the camera motion trajectory;

[0117] Converting the image feature points of each of the key frame images into an initial point cloud based on the feature point correspondence relationship and the multi-view geometric relationship;

[0118] The initial point cloud is densely reconstructed to obtain the target point cloud.

[0119] In some embodiments, after executing the step of performing dense reconstruction processing on the initial point cloud to obtain the target point cloud, the processing unit 602 is further configured to:

[0120] Constructing a plurality of second grids for the target point cloud according to a preset surface reconstruction algorithm;

[0121] The plurality of second meshes are texture mapped based on the plurality of key frame images to obtain a spatial grid model.

[0122] To summarize, the device 600 for determining the position of ball nodes in a space grid provided in an embodiment of the present application obtains a point cloud of the space grid by performing three-dimensional reconstruction of the space grid through a video of the space grid, and then obtains a spherical point cloud of each ball node in the space grid based on the point cloud of the space grid, and determines the node position of each spherical node based on the spherical point cloud of each ball node. It can be seen that the position of the ball nodes in the space grid can be obtained through this solution.

[0123] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the ball node position determination device and each unit in the above-mentioned space grid can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0124] The device for determining the position of the ball nodes in the space grid can be implemented in the form of a computer program. The computer program can be used in Figure 7 Runs on the computer device shown.

[0125] See also Figure 7 , Figure 7 700 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 700 may be a terminal or a server.

[0126] See also Figure 7 The computer device 700 includes a processor 702 , a memory and a network interface 705 connected via a system bus 701 , wherein the memory may include a non-volatile storage medium 703 and an internal memory 704 .

[0127] The non-volatile storage medium 703 can store an operating system 7031 and a computer program 7032. The computer program 7032 includes program instructions, and when the program instructions are executed, the processor 702 can execute a method for determining the position of a ball node in a space grid.

[0128] The processor 702 is used to provide computing and control capabilities to support the operation of the entire computer device 700 .

[0129] The internal memory 704 provides an environment for the operation of the computer program 7032 in the non-volatile storage medium 703. When the computer program 7032 is executed by the processor 702, the processor 702 can execute a method for determining the position of a ball node in a space grid.

[0130] The network interface 705 is used to communicate with other devices over the network. Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 700 to which the solution of the present application is applied. The specific computer device 700 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] The processor 702 is used to run the computer program 7032 stored in the memory to implement the following steps:

[0132] Acquire a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles;

[0133] Performing three-dimensional reconstruction processing on the target video to obtain a target point cloud;

[0134] Dividing the space where the target point cloud is located into a plurality of first grids according to a preset grid overlap rate, wherein the first grid is a cube with a side length equal to the diameter of the maximum spherical node;

[0135] For each first grid, extracting a grid point cloud located in the first grid from the target point cloud;

[0136] For each first grid, a sphere fitting process is performed on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and a sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds is determined as the target sphere point cloud of the first grid;

[0137] Determine at least one candidate sphere set from the plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres;

[0138] Determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set;

[0139] The current node position of each ball node in the target space grid is determined according to the target sphere set.

[0140] It should be understood that in the embodiment of the present application, the processor 702 may be a central processing unit (CPU), and the processor 702 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0141] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0142] Therefore, the present application also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps:

[0143] Acquire a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles;

[0144] Performing three-dimensional reconstruction processing on the target video to obtain a target point cloud;

[0145] Dividing the space where the target point cloud is located into a plurality of first grids according to a preset grid overlap rate, wherein the first grid is a cube with a side length equal to the diameter of the maximum spherical node;

[0146] For each first grid, extracting a grid point cloud located in the first grid from the target point cloud;

[0147] For each first grid, a sphere fitting process is performed on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and a sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds is determined as the target sphere point cloud of the first grid;

[0148] Determine at least one candidate sphere set from the plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres;

[0149] Determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set;

[0150] The current node position of each ball node in the target space grid is determined according to the target sphere set.

[0151] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0152] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0154] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0156] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for determining the position of a ball node in a space grid, characterized in that: include: Acquire a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles; Performing three-dimensional reconstruction processing on the target video to obtain a target point cloud; Dividing the space where the target point cloud is located into a plurality of first grids according to a preset grid overlap rate, wherein the first grid is a cube with a side length equal to the diameter of the maximum spherical node; For each first grid, extracting a grid point cloud located in the first grid from the target point cloud; For each first grid, a sphere fitting process is performed on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and a sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds is determined as the target sphere point cloud of the first grid; Determining at least one candidate sphere set from the plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres; Determine the candidate sphere set containing the largest number of target sphere point clouds as the target sphere set; Determining the current node position of each ball node in the target space grid according to the target sphere set; The step of performing sphere fitting processing on the grid point cloud according to the multiple candidate radii to obtain multiple candidate sphere point clouds includes: Taking the average distance between each two points in the grid point cloud as an initial fitting parameter; The initial fitting parameter is divided by a preset adjustment parameter to obtain a minimum fitting radius, and the initial fitting parameter is multiplied by the adjustment parameter to obtain a maximum fitting radius, where the adjustment parameter is a value greater than 1; Determine a radius, among the plurality of candidate radii, that is larger than the minimum fitting radius and smaller than the maximum fitting radius as a target fitting radius; Performing sphere fitting processing on the grid point cloud according to the target fitting radius to obtain a plurality of candidate sphere point clouds; The step of determining at least one candidate sphere set from a plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres comprises: Performing filtering processing on the target sphere point clouds whose fitting error values ​​are greater than the fitting error threshold among the plurality of target sphere point clouds; Determine the sphere distance between each target sphere point cloud obtained after filtering; At least one candidate sphere set is determined from the plurality of target sphere point clouds obtained after filtering according to the sphere distance and the minimum distance between spheres.

2. The method according to claim 1, characterized in that After extracting the grid point cloud located in the first grid from the target point cloud and before performing sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, the method further includes: The first grid whose number of points in the grid point cloud is less than a preset point number threshold is filtered.

3. The method according to claim 1, characterized in that: After determining the current node position of each ball node in the target space grid according to the target sphere set, the method further includes: The ball node displacement analysis is performed according to the current ball node position of each ball node and the preset original node position of each ball node to obtain the ball node displacement analysis result.

4. The method according to any one of claims 1 to 3, characterized in that The three-dimensional reconstruction of the target video to obtain a target point cloud includes: Extracting multiple key frame images from the target video according to a preset frame extraction rule; Extracting image feature points of each of the key frame images respectively, and performing feature point matching processing on a plurality of the image feature points to establish a feature point correspondence relationship between the image feature points; Based on the correspondence relationship of the feature points, a camera motion trajectory is determined using a basic matrix and an essential matrix, and a multi-view geometric relationship of multiple frames of the key frame images is determined according to the camera motion trajectory; Converting the image feature points of each of the key frame images into an initial point cloud based on the feature point correspondence relationship and the multi-view geometric relationship; The initial point cloud is densely reconstructed to obtain the target point cloud.

5. The method according to claim 4, characterized in that After performing dense reconstruction processing on the initial point cloud to obtain the target point cloud, the method further includes: Constructing a plurality of second grids for the target point cloud according to a preset surface reconstruction algorithm; The plurality of second meshes are texture mapped based on the plurality of key frame images to obtain a spatial grid model.

6. A device for determining the position of a ball node in a space grid, characterized in that: include: A transceiver unit, used to obtain a target video, wherein the target video is a video obtained by shooting the target space grid at multiple angles; A processing unit is used to perform three-dimensional reconstruction processing on the target video to obtain a target point cloud; divide the space where the target point cloud is located into multiple first grids according to a preset grid overlap rate, where the first grid is a cube with a side length equal to the maximum spherical node diameter; for each first grid, extract a grid point cloud located in the first grid from the target point cloud; for each first grid, perform sphere fitting processing on the grid point cloud according to multiple candidate radii to obtain multiple candidate sphere point clouds, and determine the sphere point cloud with the smallest fitting error among the multiple candidate sphere point clouds as the target sphere point cloud of the first grid; Based on a preset fitting error threshold, the fitting error value of each target sphere point cloud and a preset minimum distance between spheres, at least one candidate sphere set is determined from the plurality of target sphere point clouds; the candidate sphere set containing the largest number of target sphere point clouds is determined as the target sphere set; and the current node position of each sphere node in the target space grid is determined according to the target sphere set; When the processing unit performs the step of performing sphere fitting processing on the grid point cloud according to the multiple candidate radii to obtain multiple candidate sphere point clouds, it is specifically used to: Taking the average distance between each two points in the grid point cloud as an initial fitting parameter; The initial fitting parameter is divided by a preset adjustment parameter to obtain a minimum fitting radius, and the initial fitting parameter is multiplied by the adjustment parameter to obtain a maximum fitting radius, where the adjustment parameter is a value greater than 1; Determine a radius, among the plurality of candidate radii, that is larger than the minimum fitting radius and smaller than the maximum fitting radius as a target fitting radius; Performing sphere fitting processing on the grid point cloud according to the target fitting radius to obtain a plurality of candidate sphere point clouds; When executing the step of determining at least one candidate sphere set from a plurality of target sphere point clouds based on a preset fitting error threshold, a fitting error value of each target sphere point cloud, and a preset minimum distance between spheres, the processing unit is specifically configured to: Performing filtering processing on the target sphere point clouds whose fitting error values ​​are greater than the fitting error threshold among the plurality of target sphere point clouds; Determine the sphere distance between each target sphere point cloud obtained after filtering; At least one candidate sphere set is determined from the plurality of target sphere point clouds obtained after filtering according to the sphere distance and the minimum distance between spheres.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the method for determining the position of ball nodes in a space grid as described in any one of claims 1-5.

8. A storage medium, characterized in that: The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the method for determining the position of a ball node in a space grid as described in any one of claims 1-5.

Images