Three-dimensional scanning system and method, storage medium and electronic equipment

Abstract

The invention provides a three-dimensional scanning system and method, a storage medium and electronic equipment, and belongs to the field of three-dimensional scanning, and the system comprises a processing end, a control end, a scanner and at least two pattern projectors. A signal receiver is arranged on the pattern projector and is used for receiving a control signal of the control end; the projection wavelengths of the pattern projectors are different from one another; the scanner is used for scanning an object to be scanned to obtain a scanning image and transmitting the scanning image to the control end; the pattern projector is used for performing pattern projection on the surface of the to-be-scanned object; the control end is used for controlling the scanner and the pattern projectors to work cooperatively and sending the scanned images transmitted by the scanner to the processing end; and the processing end is used for carrying out three-dimensional data splicing according to the received scanning image. By applying the system provided by the invention, three-dimensional data splicing can be realized by shooting the scanning image of the projection pattern, point pasting operation is not needed, and the scanning efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional scanning system and method, a storage medium and an electronic device. Background Art

[0002] Handheld laser scanners are one of the common scanning devices in three-dimensional scanning scenarios. In the application scenarios of handheld laser scanners, it is usually necessary to splice the three-dimensional data measured at different scanning angles to construct the final three-dimensional model.

[0003] At present, it is usually done by manually deploying multiple markers with certain identification on the surface of the scanned object, and setting a tracker in the scanning scene. When scanning the object, the position of each marker is located by the tracker, and when splicing the 3D data of different scanning angles, the 3D data of different scanning angles are spliced ​​according to the positions of the same markers captured at different scanning angles.

[0004] In the existing 3D data stitching process, it is necessary to perform the operation of marking points. In actual application scenarios, objects with large surface areas or complex shapes and structures are often encountered. The point-sticking operation is usually very complicated, making the 3D scanning process more cumbersome and the scanning efficiency lower. In addition, the equipment cost of the tracker is relatively high. Summary of the invention

[0005] In view of this, embodiments of the present invention provide a three-dimensional scanning system and method, a storage medium and an electronic device to solve the problem that the existing three-dimensional scanning method relies on a tracker and marker points posted on the object to achieve three-dimensional data splicing, requires complex point-sticking operations, has low efficiency and high equipment costs.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] A three-dimensional scanning system, comprising:

[0008] A processing end, a control end, a scanner and at least two pattern projectors; each of the pattern projectors is provided with a signal receiver; the projection wavelengths of the pattern projectors are different from each other;

[0009] The scanner is used to scan the object to be scanned, obtain a scanned image, and transmit the scanned image to the control end;

[0010] Each of the pattern projectors is used to project a pattern onto the surface of the object to be scanned;

[0011] A signal receiver on each of the pattern projectors, used to receive a control signal from the control end;

[0012] The control end is used to control the scanner and each of the pattern projectors to work together, and send the scanned image transmitted by the scanner to the processing end;

[0013] The processing end is used to perform three-dimensional data splicing according to the received scanned images.

[0014] In the above system, optionally, the process of performing three-dimensional data stitching according to the received scanned image includes:

[0015] Performing three-dimensional reconstruction based on the scanned image corresponding to the current frame to obtain three-dimensional data of the current frame;

[0016] Performing projection pattern recognition based on the scanned image corresponding to the current frame to obtain projection patterns of each current frame;

[0017] Determining a pattern matching relationship between each of the current frame projection patterns and each of the other frame projection patterns that have been acquired;

[0018] Determining a relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames that have been acquired based on the pattern matching relationship;

[0019] According to the relative position relationship, data splicing is performed on the three-dimensional data of the current frame and the three-dimensional data of other frames.

[0020] Optionally, the above system, performing three-dimensional reconstruction based on the scanned image corresponding to the current frame to obtain three-dimensional data of the current frame includes:

[0021] Performing image separation on the scanned image corresponding to the current frame to obtain a first image and a second image corresponding to each of the pattern projectors; the wavelength range of the first image matches the wavelength of the laser projected by the scanner, and the wavelength range of each of the second images matches the projection wavelength of the pattern projector corresponding to the second image;

[0022] Performing laser measurement processing based on the first image to obtain the three-dimensional coordinates of each laser imaging point;

[0023] Based on the three-dimensional coordinates of each of the laser imaging points, the three-dimensional data of the current frame is determined.

[0024] In the above system, optionally, the performing of projection pattern recognition based on the scanned image corresponding to the current frame to obtain each current frame projection pattern includes:

[0025] Performing pattern recognition on each of the second images respectively to obtain a projection pattern in each of the second images;

[0026] The projection patterns in each of the second images are used as the projection patterns of each of the current frames.

[0027] In the above system, optionally, the determining of the pattern matching relationship between each of the current frame projection patterns and each of the other frame projection patterns that have been acquired includes:

[0028] For each of the current frame projection patterns, perform pattern comparison between the current frame projection pattern and each of the other frame projection patterns to obtain a pattern comparison result corresponding to the current frame projection pattern;

[0029] The pattern matching relationship is determined based on the pattern comparison results corresponding to each of the current frame projection patterns.

[0030] Optionally, in the above system, determining the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames that have been acquired based on the pattern matching relationship includes:

[0031] According to the pattern matching relationship, a target projection pattern is determined in each of the current frame projection patterns, and an associated projection pattern corresponding to the target projection pattern is determined in each of the other frame projection patterns;

[0032] Determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame;

[0033] In the other frame three-dimensional data, determining the central three-dimensional coordinates corresponding to the associated projection pattern;

[0034] The relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of the other frames is determined according to the central three-dimensional coordinates corresponding to the target projection pattern and the central three-dimensional coordinates corresponding to the associated projection pattern.

[0035] In the above system, optionally, determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame includes:

[0036] Determine a first mapping relationship between the three-dimensional data of the current frame and pixel points in a scanned image corresponding to the current frame;

[0037] Determine a second mapping relationship between a pattern center point of the target projection pattern and a pixel point in the scanned image corresponding to the current frame;

[0038] Based on the first mapping relationship and the second mapping relationship, in the three-dimensional data of the current frame, the three-dimensional coordinates mapped by the pattern center point of the target projection pattern are determined, and the three-dimensional coordinates mapped by the pattern center point of the target projection pattern are used as the center three-dimensional coordinates corresponding to the target projection pattern.

[0039] A three-dimensional scanning method, the three-dimensional scanning method is applied to a three-dimensional scanning system, the three-dimensional scanning system includes a processing end, a control end, a scanner and at least two pattern projectors, the three-dimensional scanning method includes:

[0040] Controlling each of the pattern projectors to project a pattern onto the surface of the object to be scanned; the projection wavelengths of each of the pattern projectors are different from each other;

[0041] Controlling the scanner to scan the object to be scanned, obtaining a scanned image, and transmitting the scanned image to the control end;

[0042] Sending the scanned image transmitted by the scanner to the processing end through the control end;

[0043] The processing end is enabled to perform three-dimensional data splicing according to the received scanned image.

[0044] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional scanning method as described above.

[0045] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the three-dimensional scanning method as described above.

[0046] A three-dimensional scanning system provided based on the above-mentioned embodiment of the present invention includes: a processing end, a control end, a scanner and at least two pattern projectors; each pattern projector is provided with a signal receiver; the projection wavelengths of each pattern projector are different from each other; the scanner is used to scan the object to be scanned, obtain the scanned image, and transmit the scanned image to the control end; each pattern projector is used to project the pattern onto the surface of the object to be scanned; the signal receiver on each pattern projector is used to receive the control signal of the control end; the control end is used to control the scanner and each pattern projector to work together, and send the scanned image transmitted by the scanner to the processing end; the processing end is used to perform three-dimensional data splicing according to the received scanned image. Using the system provided by the embodiment of the present invention, a plurality of pattern projectors with different projection wavelengths can be used to project the pattern onto the object to be scanned during the scanning process, and three-dimensional data splicing can be performed based on the scanned image during the data processing process. The pattern matching relationship between the projection pattern of the current frame scanned image and the projection pattern of other frame scanned images can be used to identify the key feature points of data splicing, thereby realizing three-dimensional data splicing. There is no need to perform complex point sticking operations during the scanning process, and the scanning process is relatively simple, which is conducive to improving scanning efficiency and user experience. Secondly, there is no need to deploy trackers, which helps reduce scanning costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0048] Figure 1 A schematic diagram of the system structure of a three-dimensional scanning system provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a three-dimensional data splicing process provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the deployment of a three-dimensional scanning system provided by an embodiment of the present invention;

[0051] Figure 4 A method flow chart of a three-dimensional scanning method provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0055] An embodiment of the present invention provides a three-dimensional scanning system, the structural schematic diagram of the three-dimensional scanning system can be shown as follows: Figure 1 As shown, the system includes:

[0056] A processing end 101, a control end 102, a scanner 103 and at least two pattern projectors 104; each of the pattern projectors 104 is provided with a signal receiver; the projection wavelengths of the pattern projectors 104 are different from each other;

[0057] The scanner 103 is used to scan the object to be scanned, obtain a scanned image, and transmit the scanned image to the control terminal 102;

[0058] Each of the pattern projectors 104 is used to project a pattern onto the surface of the object to be scanned;

[0059] The signal receiver on each of the pattern projectors 104 is used to receive the control signal of the control terminal 102;

[0060] The control end 102 is used to control the scanner 103 and each of the pattern projectors 104 to work together, and send the scanned image transmitted by the scanner 103 to the processing end 101;

[0061] The processing end 101 is used to perform three-dimensional data splicing according to the received scanned images.

[0062] An embodiment of the present invention provides a three-dimensional scanning system, which includes a processing end, a control end, a scanner and a plurality of pattern projectors. A signal receiver is provided on the pattern projector, and the signal receiver can be connected to the control end in a wired or wireless manner for communication. The pattern projector can project random spots and other patterns with a beam of light of a certain wavelength. The projection wavelengths of each pattern projector are different from each other, that is, the wavelengths of the beams projected by each pattern projector are different from each other. For example, two pattern projectors are provided, and the projection wavelength of one pattern projector is a red light wavelength, and the projection wavelength of the other pattern projector is a green light wavelength. The patterns projected by each pattern projector may be the same or different. During the scanning process, each pattern projector can be started to project onto the surface of the object to be scanned, and the pattern projected by the pattern projector will be projected onto the surface of the object to be scanned.

[0063] In the system provided by the embodiment of the present invention, the scanner can be a laser scanner, which can transmit a scanning signal to the control end, so that the control end controls each pattern projector to project a pattern onto the surface of the object to be scanned. The user can hold the scanner and scan the surface of the object projected by each pattern projector. When scanning through the scanner, the scanner will project a laser of a certain wavelength to the current scanning area of ​​the object to be scanned, and collect the surface image of the object in the current scanning area of ​​the object to be scanned. The collected image is the scanned image. It can be understood that the surface of the object in the scanned image collected by the scanner is projected with the laser beam emitted by the scanner and the pattern projected by the pattern projector. The scanned image collected by the scanner can be sent to the processing end through the control end. The camera on the scanner can collect patterns of different wavelengths.

[0064] The processing end can be deployed on a computing terminal, and the control end can be integrated inside a scanner or set on a computer or other peripheral device. The processing end can process the image data collected in real time during the scanning process. After receiving the corresponding scanned image, the scanned image of each frame is processed to obtain the three-dimensional data corresponding to each frame, and the three-dimensional data of each frame is spliced ​​according to the matching relationship of the projection pattern of each frame. For example, when the scanning starts and the scanned image of the first frame is collected, laser measurement can be performed based on the scanned image of the first frame to obtain the three-dimensional data corresponding to the first frame. When the scanned image of the second frame is received, laser measurement is performed based on the scanned image of the second frame to obtain the three-dimensional data corresponding to the second frame, and the three-dimensional data of the first frame and the three-dimensional data of the second frame are spliced. Similarly, when a new scanned image is obtained later, new three-dimensional data can be processed and spliced ​​with the existing three-dimensional data.

[0065] The three-dimensional scanning system provided by the embodiment of the present invention includes: a processing end, a control end, a scanner and at least two pattern projectors; each pattern projector is provided with a signal receiver; the projection wavelengths of each pattern projector are different from each other; the scanner is used to scan the object to be scanned, obtain the scanned image, and transmit the scanned image to the control end; each pattern projector is used to project the pattern onto the surface of the object to be scanned; the signal receiver on each pattern projector is used to receive the control signal of the control end; the control end is used to control the scanner and each pattern projector to work together, and send the scanned image transmitted by the scanner to the processing end; the processing end is used to perform three-dimensional data splicing according to the received scanned image. Using the system provided by the embodiment of the present invention, a pattern can be projected onto the object to be scanned by multiple pattern projectors with different projection wavelengths during the scanning process, and three-dimensional data splicing can be performed based on the scanned image during the data processing process. The pattern matching relationship between the projection pattern of the current frame scanned image and the projection pattern of other frame scanned images can be used to identify the key feature points of data splicing, thereby realizing three-dimensional data splicing. There is no need to perform complex point sticking operations during the scanning process, and the scanning process is relatively simple, which is conducive to improving scanning efficiency and user experience. Secondly, there is no need to deploy trackers, which helps reduce scanning costs.

[0066] exist Figure 1 Based on the system shown, Figure 2 As shown, in the system provided by the embodiment of the present invention, the process of performing three-dimensional data stitching by the processing end according to the received scanned image includes:

[0067] S201: performing three-dimensional reconstruction based on a scanned image corresponding to a current frame to obtain three-dimensional data of the current frame;

[0068] In the system provided by the embodiment of the present invention, when the processing end needs to process a scanned image of a certain frame, the frame can be used as the current frame, and based on the laser beam in the scanned image corresponding to the current frame, the three-dimensional coordinates of the surface of the object in the scanned image are measured by the principle of laser measurement to obtain the three-dimensional data corresponding to the current frame, that is, the three-dimensional coordinates currently measured. Specifically, the relevant data of the laser line can be directly extracted on the basis of the scanned image for three-dimensional reconstruction, or the scanned image can be further processed and three-dimensional reconstruction can be performed on the basis of the processed image, such as separating an image whose wavelength range matches the laser wavelength from the scanned image, and extracting the relevant data of the laser line on the basis of the separated image for three-dimensional reconstruction.

[0069] S202: performing projection pattern recognition based on the scanned image corresponding to the current frame to obtain projection patterns of each current frame;

[0070] In the system provided by the embodiments of the present invention, during the processing of the scanned image of the current frame, it is necessary to identify the projected pattern for the scanned image corresponding to the current frame, that is, to identify the pattern projected by the pattern projector on the object surface from the scanned image, and regard each projected pattern on the scanned image as the projected pattern of the current frame. Regarding the identification of the projected pattern of the current frame, the specifically identified information may include the image of the projected pattern of the current frame and the pattern position on the scanned image. Pattern recognition can be directly performed on the basis of the scanned image, or further image processing can be performed on the scanned image, and pattern recognition can be performed on the basis of the processed image. For example, an image with a wavelength range matching the projection wavelength of the pattern projector can be classified from the scanned image, and image analysis can be performed on the separated image to identify the pattern projected by the pattern projector.

[0071] S203: Determine the pattern matching relationship between each of the current frame projected patterns and each of the previously acquired other frame projected patterns;

[0072] In the system provided by the embodiments of the present invention, the processing end can acquire the three-dimensional data of other frames that need to be stitched with the three-dimensional data of the current frame currently, and acquire each other frame projected pattern matching the three-dimensional data of other frames. It can be understood that when the processing end processes the scanned image corresponding to other frames, three-dimensional reconstruction will be performed based on the scanned image corresponding to other frames to obtain the three-dimensional data of other frames, and projected pattern recognition will be performed based on the scanned image corresponding to other frames to obtain each other frame projected pattern, that is, the pattern projected by the pattern projector identified in the scanned image corresponding to other frames.

[0073] When processing the data of the current frame, each current frame projected pattern can be pattern-matched with each other frame projected pattern to obtain the pattern matching relationship between the current frame projected pattern and the other frame projected pattern, that is, the relationship indicating whether each current frame projected pattern matches each other frame projected pattern. Whether the current frame projected pattern matches the other frame projected pattern refers to whether the two patterns belong to the same pattern.

[0074] S204: Determine the relative position relationship between the three-dimensional data of the current frame and the previously acquired three-dimensional data of other frames according to the pattern matching relationship;

[0075] In the system provided by the embodiment of the present invention, based on the pattern matching relationship between the projection pattern of the current frame and the projection patterns of other frames, the projection patterns that appear repeatedly in the current frame scanned image and the other frame scanned images can be determined, and the center points of the repeatedly appearing projection patterns are used as key feature points. The three-dimensional coordinates corresponding to the key feature points are determined in the three-dimensional data of the current frame and the three-dimensional data of the other frames, respectively. Based on the three-dimensional coordinates corresponding to the key feature points in the current frame and the other frames, respectively, the coordinate conversion relationship between the three-dimensional data of the current frame and the three-dimensional data of the other frames, that is, the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of the other frames can be obtained.

[0076] S205: performing data splicing on the three-dimensional data of the current frame and the three-dimensional data of other frames according to the relative position relationship.

[0077] In the system provided by the embodiment of the present invention, the coordinates of the three-dimensional data of the current frame can be transformed according to the relative position relationship, and the transformed three-dimensional data can be fused with the three-dimensional data of other frames to achieve data splicing of the three-dimensional data of the current frame and the three-dimensional data of other frames.

[0078] Based on the system provided by the embodiment of the present invention, during the data processing process, three-dimensional reconstruction and projection pattern recognition can be performed based on the scanned image, and the key feature points of data splicing can be identified according to the pattern matching relationship between each projection pattern corresponding to the current frame and each projection pattern corresponding to other frames, thereby determining the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames to be spliced, and realizing three-dimensional data splicing. There is no need to perform complex point-sticking operations during the scanning process, and the scanning process is relatively simple, which is conducive to improving scanning efficiency and improving user experience. Secondly, there is no need to deploy a tracker, which is conducive to reducing scanning costs.

[0079] exist Figure 2 Based on the three-dimensional data stitching process shown, in the embodiment of the present invention, the process of performing three-dimensional reconstruction based on the scanned image corresponding to the current frame mentioned in step S201 to obtain the three-dimensional data of the current frame includes:

[0080] Performing image separation on the scanned image corresponding to the current frame to obtain a first image and a second image corresponding to each of the pattern projectors; the wavelength range of the first image matches the wavelength of the laser projected by the scanner, and the wavelength range of each of the second images matches the projection wavelength of the pattern projector corresponding to the second image;

[0081] In the system provided by the embodiment of the present invention, the scanner is a laser scanner, and the scanner has the function of shooting a spectrum or a color image, and the wavelength of the laser projected by the scanner is different from the projection wavelength of each pattern projector. In the three-dimensional reconstruction process, image separation can be performed on the scanned image corresponding to the current frame, and the scanned image can be decomposed into images of different wavelength ranges, and each image of the wavelength range is an independent image, or the scanned image can be split into channel images of different wavelength ranges of the original image, that is, after image separation is performed on the scanned image of the current frame, images of different wavelength ranges will be obtained. Each wavelength range of the image separation corresponds to the laser wavelength projected by the laser scanner and the projection wavelength of each pattern projector. Among the images obtained by separating the scanned image, the image whose wavelength range matches the laser wavelength projected by the laser scanner is used as the first image, and the image whose wavelength range matches the projection wavelength of the pattern projector is used as the second image corresponding to the pattern projector.

[0082] Performing laser measurement processing based on the first image to obtain the three-dimensional coordinates of each laser imaging point;

[0083] Based on the three-dimensional coordinates of each of the laser imaging points, the three-dimensional data of the current frame is determined.

[0084] In the system provided by the embodiment of the present invention, laser line data is extracted based on the first image, and the principle of laser triangulation is applied to calculate the three-dimensional coordinates of the object surface position corresponding to each laser imaging point in the first image, so as to obtain the three-dimensional coordinates of each laser imaging point. The three-dimensional coordinates of each laser imaging point can be directly used as the three-dimensional data of the current frame, or the three-dimensional coordinates of each laser imaging point can be further optimized and the optimized three-dimensional coordinates can be used as the three-dimensional data of the current frame.

[0085] Based on the system provided in the embodiment of the present invention, an image matching the wavelength range and the laser wavelength can be separated from the scanned image, and laser lines can be extracted on the image to complete three-dimensional reconstruction. This can reduce the mutual interference between the laser beam in the image and the projection beam of the pattern projector, improve the accuracy of three-dimensional reconstruction, and then improve the accuracy of three-dimensional scanning.

[0086] Based on the system provided in the above embodiment, in the embodiment of the present invention, the process of performing projection pattern recognition based on the scanned image corresponding to the current frame to obtain the projection pattern of each current frame mentioned in step S202 includes:

[0087] Performing pattern recognition on each of the second images respectively to obtain a projection pattern in each of the second images;

[0088] The projection patterns in each of the second images are used as the projection patterns of each of the current frames.

[0089] The system provided by the embodiment of the present invention performs image separation on the scanned image during the processing of the scanned image, and obtains images whose wavelength ranges correspond to the laser wavelength of the scanner and the projection wavelength of each pattern projector. The laser line is extracted on the image corresponding to the wavelength range of the laser wavelength, so as to perform three-dimensional reconstruction. The recognition of the projected pattern is performed on the image whose wavelength range matches the projection wavelength of the pattern projector.

[0090] Specifically, each second image obtained by separating the scanned image corresponds to each pattern projector. For each second image corresponding to the pattern projector, the projection pattern corresponding to the pattern projector is identified in the second image, and specifically, the image of the projection pattern and the location of the projection pattern in the second image can be obtained. The projection pattern identified in each second image is used as the projection pattern of the current frame.

[0091] Based on the system provided in the embodiment of the present invention, images with a wavelength range that matches the projection wavelength of the pattern projector can be separated from the scanned images, and projection patterns can be identified on these images to extract the projection pattern of the current frame and its positioning. This can reduce mutual interference between the laser beam and the projection beam of the pattern projector, improve the accuracy of pattern recognition, and thereby improve the accuracy of three-dimensional scanning.

[0092] exist Figure 2 Based on the three-dimensional data stitching process shown, in the embodiment of the present invention, the process of determining the pattern matching relationship between each of the current frame projection patterns and each of the other frame projection patterns that have been acquired in step S203 includes:

[0093] For each of the current frame projection patterns, perform pattern comparison between the current frame projection pattern and each of the other frame projection patterns to obtain a pattern comparison result corresponding to the current frame projection pattern;

[0094] In the system provided by the embodiment of the present invention, for each current frame projection pattern, the current frame projection pattern can be compared with each other frame projection pattern respectively to identify whether the other frame projection patterns and the current frame projection pattern belong to the same projection pattern, and the pattern comparison result of the current frame projection pattern and each other frame projection pattern is obtained, that is, the pattern comparison result corresponding to the current frame projection pattern, and the pattern comparison result represents whether there are other frame projection patterns that match the current frame projection pattern.

[0095] The pattern matching relationship is determined based on the pattern comparison results corresponding to each of the current frame projection patterns.

[0096] In the system provided by the embodiment of the present invention, the matching relationship between each current frame projection pattern and each other frame projection pattern can be determined according to the pattern comparison results corresponding to each current frame projection pattern. If the pattern comparison result of the current frame projection pattern shows that the current frame projection pattern and a certain other frame projection pattern belong to the same projection pattern, then it is considered that the current frame projection pattern matches the other frame projection pattern, thereby obtaining a record of the pattern matching relationship between the current frame projection pattern and the other frame projection patterns that match each other.

[0097] exist Figure 2 Based on the three-dimensional data stitching process shown, in the embodiment of the present invention, the process of determining the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames that have been acquired based on the pattern matching relationship mentioned in step S204 includes:

[0098] According to the pattern matching relationship, a target projection pattern is determined in each of the current frame projection patterns, and an associated projection pattern corresponding to the target projection pattern is determined in each of the other frame projection patterns;

[0099] In the system provided by the embodiment of the present invention, according to the pattern matching relationship, the current frame projection pattern and other frame projection patterns that match each other can be obtained. If the current frame projection pattern has other frame projection patterns that match it, the current frame projection pattern is used as the target projection pattern, and the other frame projection patterns that match it are used as their corresponding associated projection patterns.

[0100] Determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame;

[0101] In the system provided by the embodiment of the present invention, the center point of the target projection pattern is used to represent the pattern position. According to the position mapping of the target projection pattern in the scanned image of the current frame and the mapping relationship between the three-dimensional data of the current frame and the scanned image of the current frame, the three-dimensional coordinates corresponding to the center point of the target projection pattern can be found in the three-dimensional data of the current frame, that is, the three-dimensional coordinates corresponding to the center point position of the target projection pattern on the surface of the object, and the three-dimensional coordinates are used as the central three-dimensional coordinates corresponding to the target projection pattern.

[0102] In the other frame three-dimensional data, determining the central three-dimensional coordinates corresponding to the associated projection pattern;

[0103] In the system provided by an embodiment of the present invention, the center point of the associated projection pattern is used to represent the pattern position. Based on the position mapping of the associated projection pattern in other frame scanned images, as well as the mapping relationship between other frame three-dimensional data and other frame scanned images, the three-dimensional coordinates corresponding to the center point of the associated projection pattern can be found in the other frame three-dimensional data, and the three-dimensional coordinates can be used as the center three-dimensional coordinates corresponding to the associated projection pattern.

[0104] The relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of the other frames is determined according to the central three-dimensional coordinates corresponding to the target projection pattern and the central three-dimensional coordinates corresponding to the associated projection pattern.

[0105] In the system provided by the embodiment of the present invention, the target projection pattern and its corresponding associated projection pattern actually represent projection patterns at the same position. The coordinate transformation relationship between the center point of the target projection pattern and the center point of the associated projection pattern can be determined based on the central three-dimensional coordinates corresponding to the target projection pattern and the central three-dimensional coordinates corresponding to the associated projection pattern. Based on the coordinate transformation relationship, the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames can be determined.

[0106] It should be noted that in the specific implementation process, there may be only one group or multiple groups of target projection patterns and their corresponding associated projection patterns. If there are multiple target projection patterns, the relative position relationship is determined based on the central three-dimensional coordinates corresponding to all target projection patterns and the central three-dimensional coordinates of their corresponding associated projection patterns.

[0107] Based on the system provided in the above embodiment, in an embodiment of the present invention, the process of determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame includes:

[0108] Determine a first mapping relationship between the three-dimensional data of the current frame and pixel points in a scanned image corresponding to the current frame;

[0109] In the system provided by an embodiment of the present invention, the three-dimensional data of the current frame is three-dimensional data obtained by three-dimensional reconstruction based on the scanned image of the current frame. Therefore, each three-dimensional coordinate in the three-dimensional data of the current frame represents the coordinates of a point on the surface of the corresponding object in the scanned image. Therefore, the three-dimensional data of the current frame can be mapped to the scanned image to obtain a mapping relationship between each three-dimensional coordinate in the three-dimensional data of the current frame and each pixel point in the scanned image of the current frame, and this mapping relationship is used as the first mapping relationship.

[0110] Determine a second mapping relationship between a pattern center point of the target projection pattern and a pixel point in the scanned image corresponding to the current frame;

[0111] In the system provided by an embodiment of the present invention, the target projection pattern is a projection pattern identified from the current frame scan image. The scan image pixel point corresponding to the pattern center point of the target projection pattern can be determined according to the position of the target projection pattern in the current frame scan image, and the mapping relationship between the pattern center point of the target projection pattern and the pixel point in the current frame scan image is obtained, and the mapping relationship is used as the second mapping relationship.

[0112] According to the first mapping relationship and the second mapping relationship, in the three-dimensional data of the current frame, determine the three-dimensional coordinates mapped by the pattern center point of the target projection pattern, and use the three-dimensional coordinates mapped by the pattern center point of the target projection pattern as the corresponding central three-dimensional coordinates of the target projection pattern.

[0113] In the system provided by the embodiments of the present invention, based on the mapping relationship between the three-dimensional data of the current frame and the pixel points in the scanned image of the current frame, the three-dimensional coordinates mapped by each pixel point can be determined. According to the mapping relationship between the pattern center point of the target projection pattern and the pixel points of the scanned pattern, the pixel points mapped by the pattern center point can be determined. Thus, the three-dimensional coordinates corresponding to the pixel points mapped by the pattern center point can be obtained, and these three-dimensional coordinates are used as the corresponding central three-dimensional coordinates of the target projection pattern.

[0114] To better illustrate the three-dimensional scanning system provided by the embodiments of the present invention, based on the systems provided in the previous embodiments, the following further explains in combination with an actual application scenario. The deployment schematic diagram of the three-dimensional scanning system provided by the embodiments of the present invention can be as Figure 3 shown. Taking the deployment of two pattern projectors as an example, a pattern projector 1 (i.e., projector 1) and a pattern projector 2 (i.e., projector 2) are installed near the object to be scanned. The projection wavelength of pattern projector 1 is wavelength 1, and the projection wavelength of pattern projector 2 is wavelength 2. Wavelength 1 is the red light wavelength, and wavelength 2 is the green light wavelength. The scanner is a handheld laser scanner. The scanning projector of the laser scanner, that is, the laser wavelength of the laser projector is the blue light wavelength, and the filter of the laser scanner can pass signals of the three wavelengths.

[0115] If the laser scanner has a spectral separation or color photography function, the collected scanned image is separated. Images of different wavelengths are independent into one image or a channel image of a color image, such as a red channel image, a green channel image, and a blue channel image. On the image matching the projection wavelength of the pattern projector, the projection pattern and its center point are extracted to identify the key feature points for data stitching through the pattern matching relationship. On the image matching the laser wavelength, the laser line data is extracted for three-dimensional reconstruction. For example, taking the separation into different channel images as an example, that is, the extraction of the projection pattern center point is performed on the red channel image and the green channel image, and the extraction of the laser line is performed on the blue channel image for three-dimensional reconstruction.

[0116] It should be noted that if the laser wavelength of the laser scanner is the same as the projection wavelength of a certain pattern projector, or the image collected by the laser scanner does not have the basis for image separation, then three-dimensional reconstruction and pattern recognition can be directly performed on the scanned image.

[0117] The three-dimensional scanning system provided by the embodiment of the present invention does not need to perform complicated point sticking operations and does not need to be configured with a tracker, so the equipment cost is low and the operation is convenient.

[0118] The embodiment of the present invention further provides a three-dimensional scanning method, which is applied to a three-dimensional scanning system, wherein the three-dimensional scanning system includes a processing end, a control end, a scanner, and at least two pattern projectors, such as Figure 4 As shown, the three-dimensional scanning method provided by the embodiment of the present invention includes:

[0119] S301: Control each of the pattern projectors to project a pattern onto the surface of the object to be scanned; the projection wavelengths of each of the pattern projectors are different from each other;

[0120] S302: Control the scanner to scan the object to be scanned, obtain a scanned image, and transmit the scanned image to the control end;

[0121] S303: Sending the scanned image transmitted by the scanner to the processing end through the control end;

[0122] S304: enabling the processing end to perform three-dimensional data stitching according to the received scanned image.

[0123] The method provided by the embodiment of the present invention can be applied to a three-dimensional scanning system, and the architecture of the system can be as follows: Figure 1 As shown, it specifically includes a processing end, a control end, a scanner and at least two pattern projectors, and the projection wavelengths of each pattern projector are different. During the three-dimensional scanning process, each pattern projector in the three-dimensional scanning system can be controlled to project a pattern onto the surface of the object to be scanned. When the pattern is projected, the scanner is controlled to scan the object to be scanned to obtain a scanned image, and the scanned image is sent to the processing end via the control end. The processing end performs three-dimensional data stitching processing based on the received scanned image. The process of the processing end performing three-dimensional data stitching processing based on the scanned image can be referred to in conjunction with the above text. Figure 2 Regarding the specific implementation of the three-dimensional scanning process, reference may also be made to the descriptions in the various embodiments provided above for the three-dimensional scanning system, which will not be repeated here.

[0124] By applying the method provided in the embodiment of the present invention, a pattern can be projected onto the object to be scanned through multiple pattern projectors with different projection wavelengths during the scanning process, and three-dimensional data splicing can be performed based on the scanned image during the data processing process. The pattern matching relationship between the projection pattern of the current frame scan image and the projection pattern of other frame scan images can be used to identify the key feature points of data splicing, thereby realizing three-dimensional data splicing. There is no need to perform complex point sticking operations during the scanning process, and the scanning process is relatively simple, which is conducive to improving scanning efficiency and user experience. Secondly, there is no need to deploy a tracker, which is conducive to reducing scanning costs.

[0125] An embodiment of the present invention further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional scanning method as described above.

[0126] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 5 As shown, it specifically includes a memory 401 and one or more instructions 402, wherein the one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to perform the following operations:

[0127] Controlling each pattern projector to project a pattern onto the surface of the object to be scanned; the projection wavelengths of each pattern projector are different from each other;

[0128] Controlling the scanner to scan the object to be scanned, obtaining a scanned image, and transmitting the scanned image to the control end;

[0129] Sending the scanned image transmitted by the scanner to the processing end through the control end;

[0130] The processing end is enabled to perform three-dimensional data splicing according to the received scanned image.

[0131] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art may understand and implement it without creative work.

[0132] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0133] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional scanning system, characterized in that: include: A processing end, a control end, a scanner and at least two pattern projectors; each of the pattern projectors is provided with a signal receiver; the projection wavelengths of the pattern projectors are different from each other; The scanner is used to scan the object to be scanned, obtain a scanned image, and transmit the scanned image to the control end; Each of the pattern projectors is used to project a pattern onto the surface of the object to be scanned; A signal receiver on each of the pattern projectors, used to receive a control signal from the control terminal; The control end is used to control the scanner and each of the pattern projectors to work together, and send the scanned image transmitted by the scanner to the processing end; The processing end is used to perform three-dimensional data splicing according to the received scanned images.

2. The three-dimensional scanning system according to claim 1, characterized in that: The process of performing three-dimensional data stitching according to the received scanned image includes: Performing three-dimensional reconstruction based on the scanned image corresponding to the current frame to obtain three-dimensional data of the current frame; Performing projection pattern recognition based on the scanned image corresponding to the current frame to obtain projection patterns of each current frame; Determining a pattern matching relationship between each of the current frame projection patterns and each of the other frame projection patterns that have been acquired; Determining a relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames that have been acquired based on the pattern matching relationship; According to the relative position relationship, data splicing is performed on the three-dimensional data of the current frame and the three-dimensional data of other frames.

3. The three-dimensional scanning system according to claim 2, characterized in that: The three-dimensional reconstruction is performed based on the scanned image corresponding to the current frame to obtain the three-dimensional data of the current frame, including: Performing image separation on the scanned image corresponding to the current frame to obtain a first image and a second image corresponding to each of the pattern projectors; the wavelength range of the first image matches the wavelength of the laser projected by the scanner, and the wavelength range of each of the second images matches the projection wavelength of the pattern projector corresponding to the second image; Performing laser measurement processing based on the first image to obtain the three-dimensional coordinates of each laser imaging point; Based on the three-dimensional coordinates of each of the laser imaging points, the three-dimensional data of the current frame is determined.

4. The three-dimensional scanning system according to claim 3, characterized in that: The performing projection pattern recognition based on the scanned image corresponding to the current frame to obtain each current frame projection pattern includes: Performing pattern recognition on each of the second images respectively to obtain a projection pattern in each of the second images; The projection patterns in each of the second images are used as the projection patterns of each of the current frames.

5. The three-dimensional scanning system according to claim 2, characterized in that: The determining of the pattern matching relationship between each of the current frame projection patterns and each of the other frame projection patterns that have been acquired includes: For each of the current frame projection patterns, perform pattern comparison between the current frame projection pattern and each of the other frame projection patterns to obtain a pattern comparison result corresponding to the current frame projection pattern; The pattern matching relationship is determined based on the pattern comparison results corresponding to each of the current frame projection patterns.

6. The three-dimensional scanning system according to claim 2, characterized in that: Determining the relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of other frames that have been acquired based on the pattern matching relationship includes: According to the pattern matching relationship, a target projection pattern is determined in each of the current frame projection patterns, and an associated projection pattern corresponding to the target projection pattern is determined in each of the other frame projection patterns; Determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame; In the other frame three-dimensional data, determining the central three-dimensional coordinates corresponding to the associated projection pattern; The relative position relationship between the three-dimensional data of the current frame and the three-dimensional data of the other frames is determined according to the central three-dimensional coordinates corresponding to the target projection pattern and the central three-dimensional coordinates corresponding to the associated projection pattern.

7. The three-dimensional scanning system according to claim 6, characterized in that: Determining the central three-dimensional coordinates corresponding to the target projection pattern in the three-dimensional data of the current frame includes: Determine a first mapping relationship between the three-dimensional data of the current frame and pixel points in a scanned image corresponding to the current frame; Determine a second mapping relationship between a pattern center point of the target projection pattern and a pixel point in the scanned image corresponding to the current frame; Based on the first mapping relationship and the second mapping relationship, in the three-dimensional data of the current frame, the three-dimensional coordinates mapped by the pattern center point of the target projection pattern are determined, and the three-dimensional coordinates mapped by the pattern center point of the target projection pattern are used as the center three-dimensional coordinates corresponding to the target projection pattern.

8. A three-dimensional scanning method, characterized in that: The three-dimensional scanning method is applied to a three-dimensional scanning system, the three-dimensional scanning system includes a processing end, a control end, a scanner and at least two pattern projectors, and the three-dimensional scanning method includes: Controlling each of the pattern projectors to project a pattern onto the surface of the object to be scanned; the projection wavelengths of each of the pattern projectors are different from each other; Controlling the scanner to scan the object to be scanned, obtaining a scanned image, and transmitting the scanned image to the control end; Sending the scanned image transmitted by the scanner to the processing end through the control end; The processing end is enabled to perform three-dimensional data splicing according to the received scanned image.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the three-dimensional scanning method as claimed in claim 8.

10. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the three-dimensional scanning method as claimed in claim 8.

Images