Spatial 3D scanning system based on real scene

Abstract

The present invention relates to the technical field of real-scene scanning, and discloses a spatial three-dimensional scanning system based on real scenes, which specifically comprises: a long-distance scanning module, a short-distance scanning module, a texture similarity verification module, a light similarity verification module, a duplicate verification module, a three-dimensional grid generation module and a texture data projection module; the present invention solves the problems in the prior art of low scanning quality and low scanning efficiency caused by insufficient scanning accuracy, image splicing errors and inconsistent scanning light.

Description

Technical Field

[0001] The invention relates to the technical field of real scene scanning, which is a spatial three-dimensional scanning system based on real scenes. Background Art

[0002] Spatial 3D scanning systems are used to create 3D models of objects or environments. Existing spatial 3D scanning systems usually adopt a single positioning and scanning mode and cannot be flexibly adjusted according to the complexity of the target and changes in the environment. This leads to problems such as insufficient scanning accuracy, large image stitching errors, and complex data processing when scanning large or complex targets.

[0003] First, existing scanning systems usually use a single scanning mode when performing large-scale scanning, which may result in serious loss of target object details when scanning at long distances. When scanning at close range, the limited scanning range may require multiple adjustments to the device position, increasing the complexity and time cost of the operation.

[0004] Secondly, traditional scanning systems mainly rely on feature matching or texture-based methods when processing the stitching of adjacent scanned images. These methods are prone to stitching errors or reduced accuracy when the surface texture of the target object is not rich or the lighting conditions are poor. In addition, changes in ambient light, such as shadows and reflections, may also affect the accuracy of the scanned data.

[0005] Furthermore, existing 3D scanning systems often consume a lot of computing resources and time when generating 3D models, especially when processing large-scale point cloud data. The data processing efficiency is low, which limits the real-time performance and application scope of the system.

[0006] In summary, the existing spatial 3D scanning systems have obvious deficiencies in scanning accuracy, stitching accuracy, adaptability and data processing efficiency, and a more flexible, precise and efficient 3D scanning solution is needed. Summary of the invention

[0007] The technical problem to be solved by the present invention is to propose a spatial three-dimensional scanning system based on real scenes in order to solve the problems in the prior art such as insufficient scanning accuracy, image splicing errors and inconsistent scanning light leading to low scanning quality and low scanning efficiency.

[0008] In order to achieve the above object, the spatial three-dimensional scanning system based on real scene of the present invention includes the following modules:

[0009] Long-distance scanning module, short-distance scanning module, texture similarity checking module, light similarity checking module, duplicate checking module, three-dimensional mesh generation module and texture data projection module;

[0010] The long-distance scanning module is used to adjust the spatial three-dimensional scanning device to the long-distance scanning mode, perform long-distance scanning on the object to be scanned, and obtain a long-distance scanning image;

[0011] The short-range scanning module is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning;

[0012] The texture similarity checking module is used to obtain a combination of adjacent frame scanned images of a close-range scan, cut the combination of adjacent frame scanned images, import the cut adjacent frame scanned images into the texture similarity checking strategy, and execute the first scanning control strategy according to the texture similarity checking result;

[0013] The light similarity checking module is used to import the cut adjacent frame scanned images into the light similarity checking strategy, and execute the second scanning control strategy according to the light similarity checking result;

[0014] The repeated verification module is used to verify the texture similarity of the updated adjacent frame scan image combination;

[0015] The three-dimensional mesh generation module is used to extract the finally obtained close-range scanning data, and generate a three-dimensional mesh model of the target object to be scanned through the point cloud data in the close-range scanning data;

[0016] The texture data projection module is used to perform point cloud position matching on each pixel point in the close-range scanning data, and to project the texture data in the close-range scanning data onto a three-dimensional grid to form a texture map, thereby completing real-scene scanning.

[0017] Specifically, the short-range scanning module includes: a starting point position calculation unit;

[0018] The starting point position calculation unit calculates and obtains the starting point position of the short-range scan according to the long-range scan data. The specific calculation strategy of the starting point position is:

[0019] ;

[0020] in, The starting point of the close-range scan;

[0021] e is the index of all pixels in the long-distance scanned image;

[0022] is the horizontal coordinate of the e-th pixel in the long-distance scanned image;

[0023] is the vertical coordinate of the e-th pixel in the long-distance scanned image;

[0024] E is the total number of all pixels in the long-distance scanning image.

[0025] Specifically, the short-range scanning module further includes: a scanning starting point position extraction unit and a short-range scanning control unit;

[0026] The scanning starting point position extraction unit is used to extract the starting point position of the close-range scanning;

[0027] The short-range scanning control unit is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning, wherein the standard short-range scanning mode includes: configuring the standard scanning speed of the spatial three-dimensional scanning device , and ensure that in the combination of two adjacent scanned image frames, the intersection point between the next scanned image frame and the previous scanned image frame is at the center position of the previous scanned image frame, the length of the previous scanned image frame and the next scanned image frame are both CD, and the width is both KD; the standard overlap ratio of the previous scanned image frame and the next scanned image frame is 50%.

[0028] Specifically, the texture similarity checking module includes: a first adjacent frame image extraction unit, an image cutting unit and a texture similarity calculation unit;

[0029] The first adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images scanned at a close distance and implement texture similarity verification;

[0030] The image cutting unit evenly cuts the previous frame scan image and the next frame scan image in the adjacent frame scan image combination into 100 unit rectangular grids according to the unit width of the pixel block, the index of the unit rectangular grid in the previous frame scan image is n, and the index of the unit rectangular grid in the next frame scan image is m, wherein the length of the unit rectangular grid is the width KD of the previous frame scan image, and the width of the unit rectangular grid is ; The unit rectangular grid at the edge of the overlapping area between the previous frame scan image and the next frame scan image is the overlapping edge grid; in the standard close-range scanning mode, the number of non-overlapping unit rectangular grids in the next frame scan image is 50;

[0031] The texture similarity calculation unit is used to import the cut adjacent frame scanned images into the texture similarity verification strategy to calculate and obtain the first texture similarity coefficient and the second texture similarity coefficient. The texture similarity verification strategy includes the following specific steps:

[0032] B1: Convert the overlapping edge grids of the previous scanned image and the non-overlapping 50 unit rectangular grids in the next scanned image into local binary patterns, where the conversion strategy is:

[0033] ;

[0034] ;

[0035] in, and In , 2 is the neighborhood radius, and 16 is the number of pixels selected in the neighborhood;

[0036] In the previous frame scanned image, the coordinates are in the overlapping edge grid with index n. The local binary value of the pixel point after conversion;

[0037] The coordinates are In the neighborhood corresponding to the pixel point, the index of the selected pixel point is p;

[0038] The coordinates are The pixel value of the pixel point;

[0039] Represents the pixel value of the pth pixel;

[0040] In the scanned image for the next frame, the coordinates are in the non-overlapping unit rectangular grid with index m. The local binary value of the pixel point after conversion;

[0041] The coordinates are In the neighborhood corresponding to the pixel point, the index of the selected pixel point is q;

[0042] The coordinates are The pixel value of the pixel point;

[0043] Represents the pixel value of the qth pixel;

[0044] is a sign function. When the independent variable of the sign function is greater than or equal to 0, the output of the sign function is 1, otherwise it is 0;

[0045] B2: At the same time, the 50 non-overlapping unit rectangular grids in the next frame scan image are compared one by one with the overlapping edge grids in the previous frame scan image to obtain the relative entropy value, which specifically includes: , The relative entropy value obtained by comparing the non-overlapping unit rectangular grid with index m with the overlapping edge grid with index n;

[0046] I is all the gray levels in the overlapping edge grid, i is the index of the gray level;

[0047] is the probability of occurrence of gray level i among all gray levels in the overlapping edge grid with index n; Represents the probability of occurrence of gray level i among all gray levels in a non-overlapping unit rectangular grid with index m;

[0048] B3: According to B1-B2, calculate and obtain texture similarity, the texture similarity The calculation strategy is:

[0049] ;

[0050] in, is the first texture similarity coefficient;

[0051] ;

[0052] is the second texture similarity coefficient;

[0053] ;

[0054] is the second texture similarity coefficient;

[0055] Specifically, the texture similarity checking module further includes: a first scanning control triggering unit;

[0056] The first scanning control trigger unit is used to preset a total texture similarity threshold, and perform texture similarity level analysis according to the total texture similarity threshold. When the texture similarity level is qualified, the first scanning control strategy generation unit is triggered; when the texture similarity level is unqualified, the first scanning control strategy generation unit is not triggered, and the process jumps to the light similarity verification module;

[0057] The texture similarity level analysis includes: comparing the calculated texture similarity with the total texture similarity threshold; when the texture similarity is greater than or equal to the total texture similarity threshold, judging the texture similarity level to be qualified; when the texture similarity is less than the total texture similarity threshold, judging the texture similarity level to be unqualified.

[0058] Specifically, the texture similarity checking module further includes: a first scanning control strategy generating unit;

[0059] The first scanning control strategy generating unit is used to run the following calculation strategy:

[0060] C1: Preset the first texture similarity coefficient threshold and the second texture similarity coefficient threshold ;

[0061] C2: extracting a first texture similarity coefficient set and a second texture similarity coefficient set respectively formed by the first texture similarity coefficient and the second texture similarity coefficient in 50 non-overlapping unit rectangular grids in the next frame scanned image;

[0062] C3: Filter the first texture similarity coefficient set that is greater than The number of unit rectangular grids is , synchronously filter the second texture similarity coefficient set that is greater than The number of unit rectangular grids is ;

[0063] C4: According to C1-C3, the scanning speed to be updated of the spatial three-dimensional scanning device is calculated. The calculation strategy of the scanning speed to be updated is as follows:

[0064] ;

[0065] Wherein, k is the number of times the scanning speed of the spatial three-dimensional scanning device is iteratively updated; The scanning speed to be updated of the spatial three-dimensional scanning device updated in the k-th iteration.

[0066] Specifically, the texture similarity verification module also includes: a first scanning control strategy execution unit; the first scanning control strategy execution unit is used to extract and store the texture data of the overlapping area between the previous scanning image and the next scanning image in the combination of two adjacent scanning images to the cloud server, and at the same time adjust the scanning speed of the spatial three-dimensional scanning device to the scanning speed to be updated, and control the spatial three-dimensional scanning device to continue moving and continue scanning the target object to be scanned.

[0067] Specifically, the light similarity checking module includes: a second adjacent frame image extraction unit and a light similarity calculation unit;

[0068] The second adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images of close-range scanning and to verify the light similarity;

[0069] The light similarity calculation unit is used to import adjacent frame scanned images into the light similarity verification strategy to calculate and obtain light similarity. The light similarity verification strategy includes the following specific steps:

[0070] D1: Extract the first texture similarity coefficient set that is less than or equal to The unit rectangular grids of constitute the first remaining grid set. The number of unit rectangular grids in the first remaining grid set is , whose unit rectangular grid is indexed by ;

[0071] D2: Synchronously extract the second texture similarity coefficient set that is less than or equal to The unit rectangular grids of constitute the second remaining grid set. The number of unit rectangular grids in the second remaining grid set is , whose unit rectangular grid is indexed by ;

[0072] D3: Calculate and obtain the light similarity, the light similarity The calculation strategy is:

[0073] ;

[0074] in, are the first remaining grid check ratio coefficient and the second remaining grid check ratio coefficient, respectively. ;

[0075] In the first set of remaining grids, The grayscale mean of all pixels in a unit rectangular grid;

[0076] For the second set of remaining grids, The grayscale mean of all pixels in a unit rectangular grid;

[0077] is the grayscale mean of all pixels in the overlapping edge grid;

[0078] are the first remaining grid set, and The grayscale mean with the largest numerical difference and the grayscale mean with the smallest numerical difference;

[0079] are the second remaining grid set, and The numerical difference between the grayscale mean with the largest value and the grayscale mean with the smallest value.

[0080] Specifically, the light similarity checking module further includes: a second scanning control triggering unit and a second scanning control strategy execution unit;

[0081] The second scanning control trigger unit is used to preset a total light similarity threshold and perform light similarity level analysis based on the total light similarity threshold. When the light similarity level is qualified, the second scanning control strategy execution unit is triggered; when the light similarity level is unqualified, the second scanning control strategy execution unit is not triggered, the light supplement function of the spatial three-dimensional scanning device is turned on, and the spatial three-dimensional scanning device is kept at the current position, and rescans to obtain an updated scanning image of the current frame.

[0082] The light similarity level analysis includes: comparing the calculated light similarity with the total light similarity threshold, and when the light similarity is greater than or equal to the total light similarity threshold, judging the light similarity level to be qualified, and when the light similarity is less than the total light similarity threshold, judging the light similarity level to be unqualified;

[0083] The second scanning control strategy execution unit is used to adjust the scanning resolution of the spatial three-dimensional scanning device to 1.5 times the scanning resolution in the standard close-range scanning mode, and keep the spatial three-dimensional scanning device at the current position, rescanning to obtain an updated scanning image of the current frame.

[0084] It should be noted that targeted rescanning of the current local area only increases the scanning workload of this small part, avoiding repeated scanning of the entire target object, effectively improving scanning efficiency, and minimizing texture defects of the model.

[0085] Specifically, the repeated verification module is used to replace the updated scan image of the current frame with the next scan image in the adjacent frame scan image combination, and return to the texture similarity verification module to perform texture similarity verification on the updated adjacent frame scan image combination. When the texture similarity level is qualified, the loop is stopped, and the spatial three-dimensional scanning device is controlled to continue moving to continue scanning the target object to be scanned.

[0086] A storage medium stores instructions, and when a computer reads the instructions, the computer runs the real-scene-based spatial three-dimensional scanning system.

[0087] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned real-scene-based spatial three-dimensional scanning system is implemented.

[0088] Compared with the prior art, the technical effects of the present invention are as follows:

[0089] 1. The present invention determines the starting position of the short-range scanning by means of the long-range scanning module, and combines the precise scanning mode of the short-range scanning module, thereby effectively improving the scanning accuracy and coverage, and reducing the need for multiple adjustments and repeated scanning due to insufficient scanning range.

[0090] 2. The texture similarity checking module and the light similarity checking module of the present invention can detect and correct the splicing errors caused by illumination changes or texture differences by comparing the texture and light features of adjacent scanned images, thereby ensuring the continuity and consistency of the scanned images.

[0091] 3. The present invention can dynamically adjust the scanning strategy according to changes in the scanning environment, such as automatically turning on the light supplement function when the lighting conditions are poor, or adjusting the scanning speed when the texture similarity is insufficient, thereby improving the adaptability and robustness of the system in complex environments.

[0092] 4. The present invention can efficiently process large-scale point cloud data through a three-dimensional mesh generation module and a texture data projection module, quickly generate high-quality three-dimensional mesh models and texture maps, shorten data processing time, and improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0094] Figure 1 It is a structural schematic diagram of a spatial three-dimensional scanning system based on real scenes of the present invention;

[0095] Figure 2 It is a structural schematic diagram of the texture similarity checking module of the present invention;

[0096] Figure 3 It is a structural schematic diagram of the light similarity checking module of the present invention;

[0097] Figure 4 It is a schematic diagram of a scene in which the spatial three-dimensional scanning device of the present invention performs long-distance scanning on a target object. DETAILED DESCRIPTION

[0098] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0099] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0100] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0101] Example 1

[0102] For example, in this embodiment, it should be noted that in the real-scene 3D scanning process, in order to accurately capture the texture information of the target object, it is necessary to first use an image analysis algorithm to pre-process the adjacent frame scan images so that the system can automatically identify the structural features of the target object. For example, for a building scene, different structural parts such as walls, doors and windows, and roofs can be distinguished; for mechanical parts, key geometric elements such as planes, curved surfaces, holes, and threads can be identified;

[0103] Then, based on these recognition results, the adjacent frame scanned images are cut into image blocks of different scales, including macro-scale and micro-scale image blocks;

[0104] Macro-scale blocks usually cover larger target areas, such as entire walls or the main body of large mechanical components. Their purpose is to ensure the continuity of the scanned image in the overall layout. For example, taking the scanning of a historical building as an example, macro-scale cutting may divide the building facade along floors, facade partitions, etc. Each macro-scale block contains tens of square meters of image data, which is used to quickly verify whether the overall structure of the building matches between adjacent frames, avoiding large-scale errors such as floor dislocation and wall missing.

[0105] Micro-scale blocks focus on the fine textures of the target object, such as architectural decorative carvings, logos and text on the surface of mechanical parts, or fine processing textures. These micro-scale blocks are small in size and may only cover an area of ​​a few square centimeters or even smaller, but they are crucial to restore the true appearance of the target object. For example, when scanning antique furniture, micro-scale cutting can accurately separate the wood grain texture, carved patterns and other details on the furniture, so that they can be compared in depth for consistency later.

[0106] like Figure 1 As shown, the embodiment of the present invention discloses a real-scene-based spatial three-dimensional scanning system, including the following modules:

[0107] Long-distance scanning module, short-distance scanning module, texture similarity checking module, light similarity checking module, duplicate checking module, three-dimensional mesh generation module and texture data projection module;

[0108] The long-distance scanning module is used to adjust the spatial three-dimensional scanning device to the long-distance scanning mode, perform long-distance scanning on the object to be scanned, and obtain a long-distance scanning image;

[0109] The short-range scanning module includes: a starting point position calculation unit;

[0110] The starting point position calculation unit calculates and obtains the starting point position of the short-range scan according to the long-range scan data. The specific calculation strategy of the starting point position is:

[0111] ;

[0112] in, The starting point of the close-range scan;

[0113] e is the index of all pixels in the long-distance scanned image;

[0114] is the horizontal coordinate of the e-th pixel in the long-distance scanned image;

[0115] is the vertical coordinate of the e-th pixel in the long-distance scanned image;

[0116] E is the total number of all pixels in the long-distance scanning image.

[0117] The short-range scanning module is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning;

[0118] For example, Figure 4 As shown, in this embodiment, the target object to be scanned is a tree in an outdoor space;

[0119] In this embodiment, the spatial three-dimensional scanning device is an unmanned aerial vehicle device, wherein the unmanned aerial vehicle device is equipped with a camera with adjustable focal length, a laser radar, an infrared sensing system, and a gimbal.

[0120] The short-range scanning module further includes: a scanning starting point position extraction unit and a short-range scanning control unit;

[0121] The scanning starting point position extraction unit is used to extract the starting point position of the close-range scanning;

[0122] The short-range scanning control unit is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning, wherein the standard short-range scanning mode includes: configuring the standard scanning speed of the spatial three-dimensional scanning device , and ensure that in the combination of two adjacent scanned image frames, the intersection point between the next scanned image frame and the previous scanned image frame is at the center position of the previous scanned image frame, the length of the previous scanned image frame and the next scanned image frame are both CD, and the width is both KD; the standard overlap ratio of the previous scanned image frame and the next scanned image frame is 50%.

[0123] The texture similarity checking module is used to obtain a combination of adjacent frame scanned images of close range scanning, cut the adjacent frame scanned image combination, import the cut adjacent frame scanned images into the texture similarity checking strategy, and execute the first scanning control strategy according to the texture similarity checking result.

[0124] Example 2

[0125] Exemplarily, in this embodiment, a specific construction strategy of a light-texture bidirectional dynamic feedback mechanism is provided:

[0126] It should be noted that when the light check finds that uneven lighting may cause texture misjudgment, the light anomaly information, including detailed data such as light intensity deviation and shadow position, is transmitted to the texture similarity check module in real time through the system's built-in high-speed data link.

[0127] After receiving the feedback, the texture similarity verification module immediately adjusts the parameters in the similarity comparison algorithm. For example, in the strong light reflection area, the grayscale contrast weight is appropriately reduced to increase the attention to the texture shape features. In the shadow area, the image enhancement algorithm based on the illumination model is used to pre-process the texture before comparison to correct the texture misjudgment caused by light.

[0128] On the contrary, when the image difference caused by sudden change of light is verified, the texture similarity verification module will feed back the coordinates of the problem area, the amount of change of texture features and other information to the light similarity verification module. The light similarity verification module optimizes the lighting compensation measures according to the verification results. For example, when it is found that the texture of a local area is blurred due to insufficient light, the auxiliary lighting source on the scanning device is automatically controlled to turn on or adjust the brightness and angle so that the area can obtain appropriate lighting, which is convenient for subsequent texture re-verification.

[0129] Based on this fusion result, the repeated verification module can accurately focus on the problem area. It formulates adaptive rescanning instructions based on the comprehensive information of light and texture feedback. For areas that only need to adjust the light and recheck the texture, the scanning device is controlled to pause and wait for the light to be optimized before scanning again; for areas with inconsistent textures due to scanning trajectories, equipment jitter, etc., the rescanning path is accurately planned to avoid repeated scanning of error-free areas that have been accurately collected, ensuring that the final scan data is highly consistent in both light and texture dimensions, greatly improving the quality of the real-life 3D scanning model.

[0130] For example, Figure 2 As shown, the texture similarity checking module includes: a first adjacent frame image extraction unit, an image cutting unit and a texture similarity calculation unit;

[0131] The first adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images scanned at a close distance and implement texture similarity verification;

[0132] The image cutting unit evenly cuts the previous frame scan image and the next frame scan image in the adjacent frame scan image combination into 100 unit rectangular grids according to the unit width of the pixel block, the index of the unit rectangular grid in the previous frame scan image is n, and the index of the unit rectangular grid in the next frame scan image is m, wherein the length of the unit rectangular grid is the width KD of the previous frame scan image, and the width of the unit rectangular grid is ; The unit rectangular grids at the edge of the overlapping area of ​​the previous frame scan image and the next frame scan image are overlapping edge grids; in the standard close-range scanning mode, the number of non-overlapping unit rectangular grids in the next frame scan image is 50;

[0133] The texture similarity calculation unit is used to import the cut adjacent frame scanned images into the texture similarity verification strategy to calculate and obtain the first texture similarity coefficient and the second texture similarity coefficient. The texture similarity verification strategy includes the following specific steps:

[0134] B1: Convert the overlapping edge grids of the previous scanned image and the non-overlapping 50 unit rectangular grids in the next scanned image into local binary patterns, where the conversion strategy is:

[0135] ;

[0136] ;

[0137] in, and In , 2 is the neighborhood radius, and 16 is the number of pixels selected in the neighborhood;

[0138] In the previous frame scanned image, the coordinates are in the overlapping edge grid with index n. The local binary value of the pixel point after conversion;

[0139] The coordinates are In the neighborhood corresponding to the pixel point, the index of the selected pixel point is p;

[0140] The coordinates are The pixel value of the pixel point;

[0141] Represents the pixel value of the pth pixel;

[0142] In the scanned image for the next frame, the coordinates are in the non-overlapping unit rectangular grid with index m. The local binary value of the pixel point after conversion;

[0143] The coordinates are In the neighborhood corresponding to the pixel point, the index of the selected pixel point is q;

[0144] The coordinates are The pixel value of the pixel point;

[0145] Represents the pixel value of the qth pixel;

[0146] is a sign function. When the independent variable of the sign function is greater than or equal to 0, the output of the sign function is 1, otherwise it is 0;

[0147] B2: At the same time, the 50 non-overlapping unit rectangular grids in the next frame scan image are compared one by one with the overlapping edge grids in the previous frame scan image to obtain the relative entropy value, which specifically includes: , The relative entropy value obtained by comparing the non-overlapping unit rectangular grid with index m with the overlapping edge grid with index n;

[0148] I is all the gray levels in the overlapping edge grid, i is the index of the gray level;

[0149] is the probability of occurrence of gray level i among all gray levels in the overlapping edge grid with index n; Represents the probability of occurrence of gray level i among all gray levels in a non-overlapping unit rectangular grid with index m;

[0150] B3: According to B1-B2, calculate and obtain texture similarity, the texture similarity The calculation strategy is:

[0151] ;

[0152] in, is the first texture similarity coefficient;

[0153] ;

[0154] is the second texture similarity coefficient;

[0155] ;

[0156] is the average level of relative entropy of all non-overlapping unit rectangular grids.

[0157] The texture similarity checking module further includes: a first scanning control trigger unit;

[0158] The first scanning control trigger unit is used to preset a total texture similarity threshold, and perform texture similarity level analysis according to the total texture similarity threshold. When the texture similarity level is qualified, the first scanning control strategy generation unit is triggered; when the texture similarity level is unqualified, the first scanning control strategy generation unit is not triggered, and the process jumps to the light similarity verification module;

[0159] The texture similarity level analysis includes: comparing the calculated texture similarity with the total texture similarity threshold; when the texture similarity is greater than or equal to the total texture similarity threshold, judging the texture similarity level to be qualified; when the texture similarity is less than the total texture similarity threshold, judging the texture similarity level to be unqualified.

[0160] like Figure 2 As shown, the texture similarity checking module further includes: a first scanning control strategy generating unit;

[0161] The first scanning control strategy generating unit is used to run the following calculation strategy:

[0162] C1: Preset the first texture similarity coefficient threshold and the second texture similarity coefficient threshold ;

[0163] C2: extracting a first texture similarity coefficient set and a second texture similarity coefficient set respectively formed by the first texture similarity coefficient and the second texture similarity coefficient in 50 non-overlapping unit rectangular grids in the next frame scanned image;

[0164] C3: Filter the first texture similarity coefficient set that is greater than The number of unit rectangular grids is , synchronously filter the second texture similarity coefficient set that is greater than The number of unit rectangular grids is ;

[0165] C4: According to C1-C3, the scanning speed to be updated of the spatial three-dimensional scanning device is calculated. The calculation strategy of the scanning speed to be updated is as follows:

[0166] ;

[0167] Wherein, k is the number of times the scanning speed of the spatial three-dimensional scanning device is iteratively updated; The scanning speed to be updated of the spatial three-dimensional scanning device updated in the k-th iteration.

[0168] like Figure 2As shown, the texture similarity verification module also includes: a first scanning control strategy execution unit; the first scanning control strategy execution unit is used to extract and store texture data of the overlapping area between the previous scanning image and the next scanning image in the combination of two adjacent scanning images to the cloud server, and at the same time adjust the scanning speed of the spatial three-dimensional scanning device to the scanning speed to be updated, and control the spatial three-dimensional scanning device to continue moving and continue scanning the target object to be scanned.

[0169] The light similarity checking module is used to import the cut adjacent frame scanned images into the light similarity checking strategy, and execute the second scanning control strategy according to the light similarity checking result;

[0170] like Figure 3 As shown, the light similarity checking module includes: a second adjacent frame image extraction unit and a light similarity calculation unit;

[0171] The second adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images of close-range scanning and to verify the light similarity;

[0172] The light similarity calculation unit is used to import adjacent frame scanned images into the light similarity verification strategy to calculate and obtain light similarity. The light similarity verification strategy includes the following specific steps:

[0173] D1: Extract the first texture similarity coefficient set that is less than or equal to The unit rectangular grids of constitute the first remaining grid set. The number of unit rectangular grids in the first remaining grid set is , whose unit rectangular grid is indexed by ;

[0174] D2: Synchronously extract the second texture similarity coefficient set that is less than or equal to The unit rectangular grids of constitute the second remaining grid set. The number of unit rectangular grids in the second remaining grid set is , whose unit rectangular grid is indexed by ;

[0175] D3: Calculate and obtain the light similarity, the light similarity The calculation strategy is:

[0176] ;

[0177] in, are the first remaining grid check ratio coefficient and the second remaining grid check ratio coefficient, respectively. ;

[0178] In the first set of remaining grids, The grayscale mean of all pixels in a unit rectangular grid;

[0179] For the second set of remaining grids, The grayscale mean of all pixels in a unit rectangular grid;

[0180] is the grayscale mean of all pixels in the overlapping edge grid;

[0181] are the first remaining grid set, and The grayscale mean with the largest numerical difference and the grayscale mean with the smallest numerical difference;

[0182] are the second remaining grid set, and The numerical difference between the grayscale mean with the largest value and the grayscale mean with the smallest value.

[0183] By way of example, in this embodiment, it should be noted that in a real-scene scanning environment, lighting conditions have a crucial impact on texture acquisition. The light similarity verification module not only monitors basic parameters such as light intensity and uniformity, but also deeply analyzes the influence of factors such as light direction, shadow distribution, and reflection characteristics on texture presentation. For example, in an indoor scene, a strong spotlight directly shines on the surface of an object, which will cause high light reflections on the surface of the object, making it possible for texture details to be obscured or distorted in the scanned image; for example, outdoors, sunlight shines through the gaps between leaves to form mottled light and shadows, resulting in uneven lighting on the surface of the object, which brings difficulties to texture recognition.

[0184] When the texture similarity verification module is working, it will also find texture misjudgments caused by lighting problems. For example, when the light changes suddenly, the texture features such as the grayscale value and color saturation of adjacent frame images may change abnormally. It seems that the texture is inconsistent, but in fact it is caused by light interference.

[0185] like Figure 3 As shown, the light similarity checking module further includes: a second scanning control triggering unit and a second scanning control strategy execution unit;

[0186] The second scanning control trigger unit is used to preset a total light similarity threshold and perform light similarity level analysis based on the total light similarity threshold. When the light similarity level is qualified, the second scanning control strategy execution unit is triggered; when the light similarity level is unqualified, the second scanning control strategy execution unit is not triggered, the light supplement function of the spatial three-dimensional scanning device is turned on, and the spatial three-dimensional scanning device is kept at the current position, and rescans to obtain an updated scanning image of the current frame.

[0187] The light similarity level analysis includes: comparing the calculated light similarity with the total light similarity threshold, and when the light similarity is greater than or equal to the total light similarity threshold, judging the light similarity level to be qualified, and when the light similarity is less than the total light similarity threshold, judging the light similarity level to be unqualified;

[0188] The second scanning control strategy execution unit is used to adjust the scanning resolution of the spatial three-dimensional scanning device to 1.5 times the scanning resolution in the standard close-range scanning mode, and keep the spatial three-dimensional scanning device at the current position, rescanning to obtain an updated scanning image of the current frame.

[0189] The repeated verification module is used to verify the texture similarity of the updated adjacent frame scan image combination;

[0190] The repeated verification module is configured with the following strategy: replacing the updated scan image of the current frame with the next scan image in the adjacent frame scan image combination, and returning to the texture similarity verification module to perform texture similarity verification on the updated adjacent frame scan image combination. When the texture similarity level is qualified, the loop is stopped, the spatial three-dimensional scanning device is controlled to continue moving, and the scanning target is continued.

[0191] The three-dimensional mesh generation module is used to extract the finally obtained close-range scanning data, and generate a three-dimensional mesh model of the target object to be scanned through the point cloud data in the close-range scanning data;

[0192] The texture data projection module is used to perform point cloud position matching on each pixel point in the close-range scanning data, and to project the texture data in the close-range scanning data onto a three-dimensional grid to form a texture map, thereby completing real-scene scanning.

[0193] Example 3

[0194] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0195] The processor runs the above-mentioned real-scene-based spatial three-dimensional scanning system by calling the computer program stored in the memory.

[0196] This embodiment provides a computer-readable storage medium having a rewritable computer program stored thereon;

[0197] When the computer program runs on a computer device, the computer device runs the above-mentioned real-scene-based spatial three-dimensional scanning system.

[0198] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0199] It should be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and / or other information.

[0200] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through a wired network or / and a wireless network. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0201] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0202] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0203] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0204] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0205] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0206] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0207] In summary, the technical effects of the present invention are as follows:

[0208] 1. The present invention determines the starting position of the short-range scanning by means of the long-range scanning module, and combines the precise scanning mode of the short-range scanning module, thereby effectively improving the scanning accuracy and coverage, and reducing the need for multiple adjustments and repeated scanning due to insufficient scanning range.

[0209] 2. The texture similarity checking module and the light similarity checking module of the present invention can detect and correct the splicing errors caused by illumination changes or texture differences by comparing the texture and light features of adjacent scanned images, thereby ensuring the continuity and consistency of the scanned images.

[0210] 3. The present invention can dynamically adjust the scanning strategy according to changes in the scanning environment, such as automatically turning on the light supplement function when the lighting conditions are poor, or adjusting the scanning speed when the texture similarity is insufficient, thereby improving the adaptability and robustness of the system in complex environments.

[0211] 4. The present invention can efficiently process large-scale point cloud data through a three-dimensional mesh generation module and a texture data projection module, quickly generate high-quality three-dimensional mesh models and texture maps, shorten data processing time, and improve processing efficiency.

[0212] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A spatial three-dimensional scanning system based on real scenes, characterized in that: The system includes the following modules: Long-distance scanning module, short-distance scanning module, texture similarity checking module, light similarity checking module, duplicate checking module, three-dimensional mesh generation module and texture data projection module; The long-distance scanning module is used to adjust the spatial three-dimensional scanning device to the long-distance scanning mode, perform long-distance scanning on the object to be scanned, and obtain a long-distance scanning image; The short-range scanning module is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning; The texture similarity checking module is used to obtain a combination of adjacent frame scanned images of a close-range scan, cut the combination of adjacent frame scanned images, import the cut adjacent frame scanned images into the texture similarity checking strategy, and execute the first scanning control strategy according to the texture similarity checking result; The light similarity checking module is used to import the cut adjacent frame scanned images into the light similarity checking strategy, and execute the second scanning control strategy according to the light similarity checking result; The repeated verification module is used to verify the texture similarity of the updated adjacent frame scan image combination; The three-dimensional mesh generation module is used to extract the finally obtained close-range scanning data, and generate a three-dimensional mesh model of the target object to be scanned through the point cloud data in the close-range scanning data; The texture data projection module is used to perform point cloud position matching on each pixel point in the close-range scanning data, and to project the texture data in the close-range scanning data onto a three-dimensional grid to form a texture map, thereby completing real-scene scanning.

2. The real-scene-based spatial three-dimensional scanning system according to claim 1, characterized in that: The short-range scanning module includes: a starting point position calculation unit; The starting point position calculation unit calculates and obtains the starting point position of the short-range scan according to the long-range scan data.

3. The real-scene-based spatial three-dimensional scanning system according to claim 2, characterized in that: The short-range scanning module further includes: a scanning starting point position extraction unit and a short-range scanning control unit; The scanning starting point position extraction unit is used to extract the starting point position of the close-range scanning; The short-range scanning control unit is used to adjust the spatial three-dimensional scanning device to a standard short-range scanning mode, and control the spatial three-dimensional scanning device to start the short-range scanning at the starting position of the short-range scanning, wherein the standard short-range scanning mode includes: configuring the standard scanning speed of the spatial three-dimensional scanning device , and ensure that in the combination of two adjacent scanned image frames, the intersection point between the next scanned image frame and the previous scanned image frame is at the center position of the previous scanned image frame, the length of the previous scanned image frame and the next scanned image frame are both CD, and the width is both KD; the standard overlap ratio of the previous scanned image frame and the next scanned image frame is 50%.

4. The real-scene-based spatial three-dimensional scanning system according to claim 3, characterized in that: The texture similarity checking module comprises: a first adjacent frame image extraction unit, an image cutting unit and a texture similarity calculation unit; The first adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images scanned at a close distance and implement texture similarity verification; The image cutting unit evenly cuts the previous frame scan image and the next frame scan image in the adjacent frame scan image combination into 100 unit rectangular grids according to the unit width of the pixel block, the index of the unit rectangular grid in the previous frame scan image is n, and the index of the unit rectangular grid in the next frame scan image is m, wherein the length of the unit rectangular grid is the width KD of the previous frame scan image, and the width of the unit rectangular grid is ; The unit rectangular grid at the edge of the overlapping area between the previous frame scan image and the next frame scan image is the overlapping edge grid; in the standard close-range scanning mode, the number of non-overlapping unit rectangular grids in the next frame scan image is 50; The texture similarity calculation unit is used to import the cut adjacent frame scanned images into the texture similarity checking strategy to calculate and obtain the first texture similarity coefficient and the second texture similarity coefficient.

5. The real-scene-based spatial three-dimensional scanning system according to claim 4, characterized in that: The texture similarity checking module further includes: a first scanning control trigger unit; The first scanning control trigger unit is used to preset a total texture similarity threshold, and perform texture similarity level analysis according to the total texture similarity threshold. When the texture similarity level is qualified, the first scanning control strategy generation unit is triggered; when the texture similarity level is unqualified, the first scanning control strategy generation unit is not triggered, and the process jumps to the light similarity verification module; The texture similarity level analysis includes: comparing the calculated texture similarity with the total texture similarity threshold; when the texture similarity is greater than or equal to the total texture similarity threshold, judging the texture similarity level to be qualified; when the texture similarity is less than the total texture similarity threshold, judging the texture similarity level to be unqualified.

6. The real-scene-based spatial three-dimensional scanning system according to claim 5, characterized in that: The texture similarity checking module further includes: a first scanning control strategy generating unit; The first scanning control strategy generating unit is used to run the following calculation strategy: C1: Preset the first texture similarity coefficient threshold and the second texture similarity coefficient threshold ; C2: extracting a first texture similarity coefficient set and a second texture similarity coefficient set respectively formed by the first texture similarity coefficient and the second texture similarity coefficient in 50 non-overlapping unit rectangular grids in the next frame scanned image; C3: Filter the first texture similarity coefficient set that is greater than The number of unit rectangular grids is , synchronously filter the second texture similarity coefficient set that is greater than The number of unit rectangular grids is ; C4: Based on C1-C3, calculate the scanning speed to be updated of the spatial three-dimensional scanning device.

7. The real-scene-based spatial three-dimensional scanning system according to claim 6, characterized in that: The texture similarity verification module also includes: a first scanning control strategy execution unit; the first scanning control strategy execution unit is used to extract and store texture data of the overlapping area between the previous scanning image and the next scanning image in the combination of two adjacent scanning image frames to the cloud server, and at the same time adjust the scanning speed of the spatial three-dimensional scanning device to the scanning speed to be updated, and control the spatial three-dimensional scanning device to continue moving and continue scanning the target object to be scanned.

8. The real-scene-based spatial three-dimensional scanning system according to claim 7, characterized in that: The light similarity checking module comprises: a second adjacent frame image extraction unit and a light similarity calculation unit; The second adjacent frame image extraction unit is used to obtain a combination of adjacent frame scan images of close-range scanning and to verify the light similarity; The light similarity calculation unit is used to import adjacent frame scanned images into the light similarity checking strategy to calculate and obtain the light similarity.

9. The real-scene-based spatial three-dimensional scanning system according to claim 8, characterized in that: The light similarity checking module further includes: a second scanning control triggering unit and a second scanning control strategy execution unit; The second scanning control trigger unit is used to preset a total light similarity threshold, and perform light similarity level analysis according to the total light similarity threshold, and when the light similarity level is qualified, trigger the second scanning control strategy execution unit; when the light similarity level is unqualified, do not trigger the second scanning control strategy execution unit, start the light supplement function of the spatial three-dimensional scanning device, and keep the spatial three-dimensional scanning device at the current position, and re-scan to obtain an updated scanning image of the current frame; The light similarity level analysis includes: comparing the calculated light similarity with the total light similarity threshold, and when the light similarity is greater than or equal to the total light similarity threshold, judging the light similarity level to be qualified, and when the light similarity is less than the total light similarity threshold, judging the light similarity level to be unqualified; The second scanning control strategy execution unit is used to adjust the scanning resolution of the spatial three-dimensional scanning device to 1.5 times the scanning resolution in the standard close-range scanning mode, and keep the spatial three-dimensional scanning device at the current position, rescanning to obtain an updated scanning image of the current frame.

10. The real-scene-based spatial three-dimensional scanning system according to claim 9, characterized in that: The repeated verification module is configured with the following strategy: replacing the updated scan image of the current frame with the next scan image in the adjacent frame scan image combination, and returning to the texture similarity verification module to perform texture similarity verification on the updated adjacent frame scan image combination. When the texture similarity level is qualified, the loop is stopped, the spatial three-dimensional scanning device is controlled to continue moving, and the scanning target is continued.

Images