Data processing apparatus, data processing method, and program

Abstract

The purpose of the present invention is to provide a data processing device, a data processing method, and a program that make it possible to compress 3D point group data while maintaining the accuracy of restored coordinates at a level that allows for use in communication equipment inspection.　This data processing device partitions measured point group data by equipment on the basis of an identification function, generates respective image files for the partitioned point group data treating each coordinate value as a color signal, performs image compression processing on the image files using compression ratios defined for the respective equipment, and saves the image files that have undergone the image compression processing in association with parameters used for the image compression processing.

Description

Technical Field

[0001] The present disclosure relates to a technique for compressing 3D point cloud data as an image file or a video file.

Background Art

[0002] In recent years, equipment inspection technologies using 3D laser scanners have been developed, enabling efficient equipment inspection. For the inspection of communication equipment, there is a technique that uses 3D point cloud data obtained using an MMS (Mobile Mapping System) or a stationary 3D laser scanner. This technique extracts communication equipment from the acquired 3D point cloud data and creates a 3D model to calculate equipment information such as pole deflection and visualize the equipment status. If two 3D point cloud data can be stored in a database and overlaid on the screen, on-site inspection will not be required, and more efficient equipment inspection and maintenance will be possible.

[0003] However, the 3D point cloud data obtained by a 3D laser scanner has x, y, and z coordinates, and when measuring a wide range or under high-precision conditions, the data size becomes extremely large. For example, the data size of a 3D point cloud is approximately 15 GB for data measured under the conditions of a speed of 30 km / h and a measurement distance of 1 km in an MMS, and approximately 165 MB for data measured under the conditions of 500,000 points per second and a measurement angle of 15° (assuming the measurement of one pole) in a stationary 3D laser scanner. Assuming that all the utility poles in Japan are acquired, under the above measurement conditions, it would be approximately 1.8 PB in the MMS (converted assuming the total length of roads in Japan is 1,256,607 km), and approximately 5.4 PB in the stationary point cloud (converted assuming the total number of utility poles is 35 million), and the equipment construction cost and running cost for storing all the data would be high.

[0004] Therefore, in order to inexpensively store the 3D point cloud data acquired throughout Japan in a database, data compression is required. In order to significantly compress the 3D point cloud data, there is a method of pseudo-converting the x, y, and z coordinates of the 3D point cloud into color signals as in Non-Patent Document 1 and using H.265 / HEVC (High Efficiency Video Coding) standardized by the international standards organization for compression.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The capacity of 3D point cloud data acquired by MMS or fixed three-dimensional (3D) laser scanners is extremely large, and the equipment construction cost and running cost for storage and maintenance become high. Therefore, data compression is required. Non-Patent Document 1 pseudo-converts the coordinates of the 3D point cloud into YUV signals of color signals and uses the technology of image compression, which is irreversible compression, to compress the acquired point cloud data all at once.

[0007] However, since the compression method of Non-Patent Document 1 compresses the entire 3D point cloud data all at once, the compression rate cannot be adjusted for each piece of equipment, and when the compression rate is increased, the coordinate positions of all 3D point clouds change. When the coordinates of the 3D point cloud change significantly due to compression, the equipment information calculated by 3D model creation is different, and there is a problem that it becomes difficult to use for communication equipment inspection technology.

[0008] Therefore, an object of the present invention is to provide a data processing apparatus, a data processing method, and a program that can compress 3D point cloud data while maintaining the accuracy of restored coordinates to a degree that can be used in communication facility inspection technology in order to solve the above problems.

Means for Solving the Problems

[0009] In order to achieve the above object, the data processing apparatus according to the present invention divides the measured point cloud data for each facility based on an identification function, generates an image file in which each coordinate value is regarded as a color signal for each of the divided point cloud data, performs image compression processing on the image file at a compression rate determined for each facility, and stores the image file subjected to the image compression processing in association with the parameters used for the image compression processing.

[0010] Specifically, the data processing apparatus according to the present invention is a data processing apparatus that processes 3D point cloud data representing the 3D coordinates of points on the surface of an outdoor structure acquired by a 3D laser scanner, an image conversion unit that generates an image file in which the 3D coordinates of each point of the 3D point cloud data are regarded as color signals for each outdoor structure, an image compression unit that performs image compression processing on each of the image files at a compression rate determined for each type of the outdoor structure corresponding to the image file, and a storage unit that stores the compressed image file subjected to the image compression processing in association with the parameters used for the image compression processing, is provided.

[0011] Further, the data processing method according to the present invention is a data processing method that processes 3D point cloud data representing the 3D coordinates of points on the surface of an outdoor structure acquired by a 3D laser scanner, generating an image file in which the 3D coordinates of each point of the 3D point cloud data are regarded as color signals for each outdoor structure, performing image compression processing on each of the image files at a compression rate determined for each type of the outdoor structure corresponding to the image file, and Storing the compressed image file obtained by performing the image compression process in association with the parameters used in the image compression process. Perform.

[0012] When performing the image compression process, the present data processing apparatus and method thereof perform compression at a compression rate corresponding to the type of outdoor structure. Therefore, the accuracy of the restored coordinates required for each type of outdoor structure can be maintained. Accordingly, the present invention can provide a data processing apparatus and a data processing method capable of compressing 3D point cloud data while maintaining the accuracy of the restored coordinates to a degree applicable to communication facility inspection technology.

[0013] The data processing apparatus according to the present invention further includes a restoration unit that uses the compressed image file and the parameters stored in the storage unit to convert the compressed image file into points of three-dimensional coordinates to generate restored point cloud data for each outdoor structure.

[0014] The image conversion unit of the data processing apparatus according to the present invention is characterized in that it converts the numerical value of the color signal corresponding to the three-dimensional coordinates of one point into a binary number, and creates a plurality of image files from the image file for every 8 bits starting from the most significant bit of the binary number. By converting the 3D point cloud data into a plurality of image files, accurate coordinate restoration becomes possible.

[0015] The image conversion unit of the data processing apparatus according to the present invention (1) Calculating a divisor of the number of point clouds included in the three-dimensional point cloud data for each outdoor structure. (2A) When the number of point clouds is 8 or more and not a prime number, let the number of point clouds be x. (2B) When the number of point clouds is less than 8 or is a prime number, so as to have a divisor other than 1 and the number of point clouds, (a) Assigning dummy data to the three-dimensional point cloud data for each outdoor structure so that the sum of the number of point clouds and the number of dummy data is x, or (b) Deleting some points from the three-dimensional point cloud data for each outdoor structure and setting the number obtained by subtracting the number of deleted points from the number of point clouds as x, and (3) Creating the image file of the pixels that can be represented by two divisors of x characterized by When the number of points in the 3D point cloud data is a prime number, normal compression cannot be performed. Therefore, compression is made possible by deleting points or adding dummy points.

[0016] The image compression unit of the data processing apparatus according to the present invention performs batch video compression processing on a plurality of the image files generated from the outdoor structures that are of the same type and exist at different locations to obtain the compressed image file (compressed video file). The 3D point cloud data of outdoor structures of the same type is similar. Therefore, higher compression becomes possible by treating these data as a video.

[0017] When video compression is performed, the restoration unit extracts an arbitrary one of the image files from the compressed image file that is the compressed video, converts it into points of three-dimensional coordinates, and generates restored point cloud data for each outdoor structure.

[0018] The present invention is a program for causing a computer to function as the data processing apparatus. The data collection apparatus of the present invention can also be realized by a computer and a program, and it is also possible to record the program on a recording medium or provide it through a network.

[0019] In addition, the above-mentioned inventions can be combined as much as possible.

Effects of the Invention

[0020] The present invention can provide a data processing apparatus, a data processing method, and a program that can compress 3D point cloud data while maintaining the accuracy of the restored coordinates to a degree that can be used in communication facility inspection technology.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0022] Embodiments of the present invention will be described with reference to the accompanying drawings. The embodiments described below are examples of the present invention, and the present invention is not limited to the following embodiments. In the present specification and drawings, components having the same reference numerals indicate the same components.

[0023] (Embodiment 1) FIG. 1 is a diagram for explaining the data processing apparatus 301 of the present embodiment. The data processing apparatus 301 is a data processing apparatus that processes three-dimensional point cloud data representing the three-dimensional coordinates of points on the surface of the outdoor structure 12 acquired by the three-dimensional laser scanner 11, For each outdoor structure 12, an image conversion unit 31 that generates an image file in which the three-dimensional coordinates of each point of the three-dimensional point cloud data are regarded as color signals, An image compression unit 34 that performs image compression processing on each of the image files at a compression rate determined for each type of outdoor structure 12 corresponding to the image file, A storage unit 35 that stores the compressed image files subjected to the image compression processing in association with the parameters used for the image compression processing, is provided.

[0024] Further, the data processing device 301 further includes a restoration unit 36 that uses the compressed image files and the parameters stored in the storage unit 35 to convert the compressed image files into points in three-dimensional coordinates to generate restored point cloud data for each outdoor structure 12. Further, the data processing device 301 includes an identification processing unit 30 and a display unit 37. The image conversion unit 31 includes a coordinate identification unit 112 and an image creation unit 113. The outdoor structure 12 is communication equipment to be inspected, such as a utility pole, a cable, a transformer, etc.

[0025] The identification processing unit 30 separates the 3D point cloud data measured by the 3D laser scanner 11 into point clouds for each type of outdoor structure 12. The image creation unit 33 of the image conversion unit 31 converts the 3D point cloud data separated for each type of outdoor structure 12 into an image file. The coordinate identification unit 32 calculates the coordinate information required when pseudo-converting coordinates into color signals. The image compression unit 34 compresses the image file at a compression rate determined for each type of outdoor structure 12. The storage unit 35 stores the compressed image file and the parameters used for image compression. The restoration unit 36 performs inverse conversion of the color signal into coordinates from the image file and the parameters stored in the storage unit 35. The display unit 37 displays the 3D point cloud data inverse-converted from the color signal into coordinates.

[0026] [Identification Processing Unit] FIG. 2 is a diagram for explaining a method of identifying an arbitrary object from a point cloud performed by the identification processing unit 30. The identification plane 21 in the present embodiment refers to a linear boundary plane used for identifying an object and is the boundary plane between class 1 and class 2. Here, a specific example will be described. Let class 1 be a utility pole and class 2 be a scaffolding bolt. By deriving the identification plane 21 through machine learning based on learning data, it is possible to classify into two groups: the point cloud constituting the utility pole and the point cloud constituting the scaffolding bolt. The coordinates S of the 3D point cloud data of the total number of points N are shown below.

Equation

[0027] As an example, a method of obtaining the identification function f(x) for the acquired point cloud and deriving the identification plane 21 using a perceptron will be described. When the input data is A=(a 1 ,a 2 ,a 3 ,···,a N ) the identification function f(a) is expressed by the following equation. [Equation 2.2] f(a)=w 0 +a 1 w 1 +a 2 w 2 +a 3 w 3 +···+a N w N Here, W=(w 0 ,w 1 ,w 2 ,w 3 ,···,w N ) refers to the weights for each feature amount.

[0028] Class 1 is the region where f(a) is greater (f(a)>0), class 2 is the region where f(a) is smaller (f(a)<0), the point cloud of class 1 is S c1 , and the point cloud of class 2 is S c2 . The input data A=(x n ,y n ,z n) Calculate the discriminant function f(a) and check whether the points included in each of Class 1 and Class 2 can be correctly discriminated. Assume that when the points of Class 1 are input and f(a)>0, and when the points of Class 2 are input and f(a)<0, it means that the discrimination is correct.

[0029] If, as a result of the calculation, Class 1 is misclassified, that is, when f(a)<0 when the points of Class 1 are input, [Equation 2.3] w’=w+ηA If Class 2 is misrecognized, that is, when f(a)>0 when the points of Class 2 are input, [Equation 2.4] w’=w-ηA According to the above two equations, the weights of the discriminant function are corrected until all data are correctly discriminated. η represents the learning rate. Finally, the discriminant function f’(a) with the corrected weights and the discriminant surface f’(a)=0 can be derived. By this method, objects can be discriminated from the point cloud based on the training data.

[0030] Figure 2 shows an example where there are two discriminant objects. By increasing the linear discriminant surface, the number of discriminable classes can be increased, and the number of discriminant objects can be increased. For example, when one discriminant surface is increased and the number of classifiable classes is three, by setting Class 1 as a utility pole, Class 2 as a scaffold bolt, and Class 3 as a cable, the point cloud can be classified into three types of facilities. From the above, by preparing multiple discriminant surfaces, it is possible to classify the acquired point cloud into point clouds for each facility to be discriminated.

[0031] [Image conversion unit, image compression unit, restoration unit] Figure 3 is a diagram for explaining the image file creation operation performed by the image conversion unit 31, the compression operation performed by the image compression unit 34, and the point cloud restoration operation performed by the restoration unit 36.

[0032] The 3D point cloud data acquired by the 3D laser scanner 11 has coordinates (x, y, z) (step S01). On the other hand, both the image file and the (R, G, B) or (Y, U, V) color signal are available. Since both the coordinates and the color signal have three variables, they can be mutually converted. The image conversion unit 31 pseudo-converts the coordinates (x, y, z) into an RGB signal or a YUV signal (YC b C r signal) and creates an image file P01 in an arbitrary format (such as PNG or TIFF) (step S02). The image compression technology determines whether to accept the RGB signal or the YUV signal as the input signal for each technology. Therefore, the image conversion unit 31 determines which signal to use to create the image file based on the image compression technology of the image compression unit 34. Note that the image compression technology of the image compression unit 34 includes not only the compression technology for still images but also the compression technology for videos described later.

[0033] The image compression unit 34 compresses the image file P01 using an arbitrary compression technology and generates a compressed image file P02 (step S03). Note that the compressed image file P02 includes not only the file obtained by compressing a still image but also the file obtained by compressing a video described later. The storage unit 35 saves the compressed image file P02 and the parameters at the time of compression, thereby saving the 3D point cloud data (step S04). Restoration from the image file to the 3D point cloud data is performed in the reverse procedure of the compression. Specifically, when it is desired to display the 3D point cloud on the display unit 37, the restoration unit 36 retrieves the compressed image file P02 and the parameters from the storage unit 35, restores the image file P01 using the arbitrary compression technology, and restores the coordinates (x, y, z) from the image file P01 (steps S05, S06).

[0034] [Image conversion unit] FIG. 4 is a flowchart for explaining the operation of the image conversion unit 31 that converts the point cloud coordinates (x, y, z) into a color signal and creates an image file from the calculated color signal. The image conversion unit 31 includes a coordinate identification unit 32 and an image creation unit 33. The coordinate identification unit 32 performs operations from step S11 to step S14, and the image creation unit 33 performs operations from step S15 to step S17.

[0035] Here, an example of converting the point group coordinates S included in Class 1 described in FIG. 3 c1 into the color signal C will be described. The point group coordinates S c1 are converted into the color signal C by the following formula. [Equation 4.1] 0 = P·S c1min +Q [Equation 4.2] 2 8 -1 = P·S c1max +Q Here, P and Q are constants of a linear function for converting coordinate values into a color signal. P is the slope of the linear function, and Q is the intercept of the linear function.

[0036] Solving this system of simultaneous equations and using the derived P and Q, any coordinate S c1n can be converted into the color signal C by the following formula. These P and Q are the aforementioned parameters. [Equation 4.3] C = P·S c1n +Q Here, S c1max represents the maximum value of each of the point group coordinates (x, y, x) included in the 3D point cloud data, and S c1min represents the minimum value of each of the point group coordinates (x, y, x) included in the 3D point cloud data, and the color signal C = (R, G, B).

[0037] For example, if S c1max is set to x c1max , S c1min is set to x c1min , S c1n is set to x c1n , and C is set to R, the x coordinate can be converted into the color signal R. By performing similar calculations for the y and z coordinates, they can be converted into the color signals G and B.

[0038] That is, first, the coordinate identification unit 32 extracts the maximum and minimum values of each of the point group coordinates (x, y, z) from the 3D point group data segmented by the identification processing unit 30 for each facility (step S11). Next, the coordinate identification unit 32 substitutes the maximum and minimum values of each coordinate into formula (4.1) and formula (4.2) to calculate the slope P and the intercept Q of the linear equation of formula (4.3) (step S12). Since the slope P and the intercept Q are necessary when restoring the point group, they are stored in the storage unit 35 (step S14). Then, the coordinate identification unit 32 converts the coordinates (x, y, z) of each point included in the point group into RGB signals using formula (4.3) (step S13).

[0039] When the image compression technology used by the image compression unit 34 accepts the input of RGB signals (Yes in step S15), the image creation unit 33 creates an image file in an arbitrary format using the RGB signals obtained by converting the point group coordinates (step S17). On the other hand, when the image compression technology used by the image compression unit 34 accepts the input of YUV signals (No in step S15), the image creation unit 33 converts the RGB signals obtained by converting the point group coordinates into YUV signals (step S16). There are several conversion formulas from RGB signals to YUV signals, and an example is shown below. [Number] The image creation unit 33 creates an image file using the values converted into YUV signals by this formula (step S17). In this way, the image creation unit 33 creates an image file in a format corresponding to the image compression technology of the image compression unit 34.

[0040] In the above example, in formula (4.2), when S max the solution of the equation is 2 8 -1, so that the maximum value of the converted number becomes 2 8 -1 = 255. This is because the color signal RGB is represented by 8 bits (0 to 255). That is, since the values that can be converted for the point group coordinates having a wide range of numbers from the minimum value to the maximum value become 256 values from 0 to 255, there will be a limitation on the coordinates that can be expressed during restoration.

[0041] Therefore, it is preferable that the image conversion unit 31 converts the numerical value of the color signal corresponding to the three-dimensional coordinates of one point into a binary number, and creates a plurality of image files for every 8 bits from the most significant bit of the binary number. This will be specifically described below.

[0042] In order to more accurately restore the point cloud coordinates, the value that can be converted for the point cloud coordinates is set to 2 8 to 2 16 , 2 24 , ···, 2 8i (i is an integer of 2 or more). When the value that can be converted for the point cloud coordinates is a number larger than 2 8 , since the color signal RGB is represented by 8 bits, it is necessary to divide the converted number into 8-bit segments. For example, when the value that can be converted for the point cloud coordinates is 2 16 -1, after converting the converted decimal value into a binary number, it is divided into the most significant 8 bits and the least significant 8 bits, and each 8-bit value is regarded as the color signal RGB, and an image file composed of the most significant 8 bits and a file composed of the least significant 8 bits are created and saved respectively.

[0043] That is, if the original RGB signal is (R 16ビット , G 16ビット , B 16ビット ), it is divided into (R 上8ビット , G 上8ビット , B 上8ビット ) and (R 下8ビット , G 下8ビット , B 下8ビット ), and image files are created respectively. By doing so, the value that can be converted for the point cloud coordinates becomes 65,536 (= 2 16 pieces) from 0 to 65,535, so that the coordinates can be restored more accurately compared to the formula using 2 8 at the time of restoration.

[0044] From this, when the value that can be converted for the point cloud coordinates is 2 8i (i = 1, 2, 3, ···), after converting the decimal value into a binary number, it is divided into i pieces of 8 bits each, and by creating an image file for each, the number that can be converted for the point cloud coordinates is 2 8iIt is possible to set it to a certain number. However, if the number that can be used for converting the point group coordinates is 2 8i pieces, the number of image files that need to be created will increase to i pieces, and the compression rate of the 3D point cloud data will deteriorate. Therefore, since there are some outdoor structures that do not require high-precision restoration accuracy depending on the type, the value 2 8i of i for converting the point group coordinates can be freely selected for each facility.

[0045] The image conversion unit 34 (1) calculates the divisors of the number of points included in the three-dimensional point cloud data for each outdoor structure 12; (2A) When the number of points is 8 or more and not a prime number, let the number of points be x; (2B) When the number of points is less than 8 or is a prime number, so as to have a divisor other than 1 and the number of points, (a) add dummy data to the three-dimensional point cloud data for each outdoor structure 12 so that the sum of the number of points and the number of dummy data is x, or (b) delete some points from the three-dimensional point cloud data for each outdoor structure 12, and let the number obtained by subtracting the number of deleted points from the number of points be x, and (3) create the image file of the pixels that can be represented by two divisors of x is characterized by.

[0046] The image creation unit 34 creates an image file such that the number of acquired points x becomes x = a × b pixels. a and b are divisors of x, and they must be numbers of 8 or more according to the method of the image compression technology. However, in fact, there are cases where the divisors of x have only 1 and x (x is a prime number). An image file of 1 × x pixels cannot be normally compressed when compressing. For this reason, when the outdoor structure 12 does not require high-precision restoration accuracy, the image creation unit 34 deletes the point cloud. On the other hand, when the outdoor structure 12 requires high-precision restoration accuracy, the image creation unit 34 adds dummy point clouds to the 3D point cloud data so that x is 8 or more and has a divisor other than 1 and x for adjustment.

[0047] FIG. 5 is a flowchart for explaining the compression operation of the image file of the still image performed by the image compression unit 34. The image compression unit 34 determines which point cloud image file of the outdoor structure 12 (step S21). For example, the image compression unit 34 determines the type of the outdoor structure 12 for each image file using the determination result by the identification processing unit 30. Then, the image compression unit 34 compresses the image file at a compression rate determined in advance for each type of the outdoor structure 12 to generate a compressed image file (step S22). In this way, the image compression unit 34 can identify the identified outdoor structure and adjust the restoration accuracy by changing the compression rate for each type of the outdoor structure.

[0048] Specifically, when the outdoor structure 12 does not require high-precision restoration accuracy, by setting a high compression rate for the image, the restoration accuracy deteriorates, but the data capacity can be reduced. On the other hand, when the outdoor structure 12 requires high-precision restoration accuracy, by setting a low compression rate for the image, the effect of reducing the data capacity becomes small, but the restoration accuracy can be maintained. For the outdoor structure 12 to be inspected, the compression rate of the image file is set so that the equipment state value (measurement value, etc.) in the inspection using the point cloud before compression and the equipment state value (measurement value, etc.) in the inspection using the point cloud obtained by restoring the image compression are not different.

[0049] In addition to diverting technologies standardized internationally such as jpeg and png as the image compression technology, the image compression unit 34 can use the entire image compression technology. As an example, the compression theory of jpeg will be described. First, the RGB signal of the input image file is converted into a YUV signal. Compression is performed by thinning out the color difference components of the YUV signal. Examples of the ratio of YUV include 4:4:4, 4:2:2, 4:1:1, and 4:2:0, and the one with 4:4:4 results in the highest quality image. After thinning out the color difference components, discrete cosine transform (DCT transform) is independently performed on each signal of the YUV signal. When discrete cosine transform is performed on the image signal f(a, b) when the YUV signal is divided into A×B pixels each, it is represented by the following formula.

Equation

[0050] By dividing the F(v, w) calculated by DCT transformation by the quantization table Q(v, w), the quantized data F q (v, w) can be calculated. [Number] By adjusting the value of α, the quantized data can be adjusted, and the compression ratio can also be set to an arbitrary ratio. After that, it is entropy encoded, multiplexed by bit string processing, and then output as a jpeg format.

[0051] In the above process, by setting an appropriate compression ratio, it is possible to compress the 3D point cloud data without causing problems in the inspection of the outdoor structure 12 using the point cloud. For example, when the compression ratio α can be selected in 10 steps (the smaller the number, the higher the quality and the lower the compression), the image file of the point cloud required for model creation can set the compression ratio α to 1, and the image file of the point cloud not required for model creation can set the compression ratio α to 10, so that it is possible to increase the compression ratio of the entire 3D point cloud data while maintaining the calculation accuracy of the equipment state by the equipment inspection technology using the point cloud.

[0052] FIG. 6 is a flowchart for explaining an operation of compressing an image file by an image compression technique (hereinafter sometimes referred to as “moving image compression technique”) in which an image compression unit 34 compresses a moving image. The image compression unit 34 determines which point cloud image file of the outdoor structure 12 (step S31). For example, the image compression unit 34 determines the type of the outdoor structure 12 for each image file using the determination result by the identification processing unit 30. Then, the image compression unit 34 compresses the image file at a compression rate determined in advance for each type of the outdoor structure 12 (step S32). Here, since the image compression unit 34 uses the moving image compression technique, a moving image of 0 seconds having an I frame, a P frame, and a B frame is generated from one image file (for each outdoor structure 12). The image compression unit 34 extracts the I frame from the moving image to obtain a compressed image file (step S33). In this way, even when the image compression unit 34 uses the moving image compression technique, it can discriminate the identified outdoor structure and adjust the restoration accuracy by changing the compression rate for each type of the outdoor structure, similarly to the image compression technique described with reference to FIG. 5.

[0053] Taking into account the restoration accuracy and data capacity required by the outdoor structure 12, set the compression rate of the moving image so that the equipment state value (such as a measurement value) in the inspection using the point cloud before compression is the same as the equipment state value (such as a measurement value) in the inspection using the point cloud obtained by restoring the image compression.

[0054] In addition to technologies standardized internationally such as MPEG-4 as the moving image compression technique, the image compression unit 34 can use the entire moving image compression technique. The moving image compression technique has functions such as DCT conversion and quantization, similar to the image compression technique, but has specific functions such as motion detection and inter-frame prediction. These are techniques for specifying a moving portion in consecutive images and compressing information other than the moving portion. By inter-frame prediction, when one image file is compressed, a moving image file of 0 seconds having an I frame, a P frame, and a B frame is created. By extracting the I frame from there, a compressed image file compressed by diverting the moving image compression technique can be obtained. By saving the compressed image file, it becomes possible to save the compressed 3D point cloud data.

[0055] As with the explanation of the image compression technology in Figure 5, by setting an appropriate compression rate α, it is possible to compress 3D point cloud data without causing any problems when inspecting the outdoor structure 12 using the point cloud.

[0056] The explanation of 3D point cloud data compression using video compression technology above gives an example of extracting and saving a target image (I frame) from a video created from a single image file. Due to the nature of video compression technology, greater compression effects can be expected when previous and next image files are similar. Since the coordinate arrangement of 3D point cloud data is similar for each piece of equipment, creating an image file for each piece of equipment will result in similar images. For this reason, it is possible to save a more highly compressed file by applying video compression technology to the image files for each piece of equipment and saving the compressed video files.

[0057] Therefore, it is preferable that the image compression unit 34 performs video compression processing on multiple image files generated from outdoor structures 12 of the same type and located in different locations at once to generate the compressed image file (compressed video file).

[0058] In this case, the restoration unit 36 ​​is characterized in that it extracts any of the image files from the compressed image file, which is a compressed video, converts them into three-dimensional coordinate points, and generates restored point cloud data for each of the outdoor structures. For example, when shooting with the MMS, the data processing device 301 divides the point clouds acquired in different scenes 1, 2, and 3 by equipment, and applies a video compression technique to the three image files of the utility poles that have been created. This makes it possible to save the three image files as one video file. In this case, the data processing device 301 needs to store, as a parameter in the storage unit 35, information about which frame in the video the image file from which point cloud coordinates are to be restored is saved, and extract the image file based on that information when necessary.

[0059] FIG. 7 is a flowchart for explaining the operation of restoring point cloud information from a compressed image file performed by the restoration unit 36. The restoration unit 36 performs inverse transformation in the reverse procedure of the flowchart of FIG. 4 to restore the point cloud. First, the restoration unit 36 extracts a color signal from the compressed image file (step S41). When a YUV signal is extracted (No in step S42), the restoration unit 36 performs inverse transformation from the mathematical formula (4.4) to convert it into an RGB signal (step S43). When an RGB signal is extracted (Yes in step S42), the restoration unit 36 proceeds directly to the next calculation (step S44). The restoration unit 36 reads the coefficients P and Q used in the mathematical formula (4.3) calculated by the image conversion unit 31 from the storage unit 35 (step S44). Then, the restoration unit 36 restores the point cloud coordinates (x, y, z) by performing inverse transformation on each RGB signal (step S45).

[0060] The display unit 37 can display the restored point cloud. Since the restoration unit 36 restores the point cloud for each outdoor structure 12, when displaying as one image, the display unit 37 needs to synthesize the restored point clouds respectively.

[0061] In the flowchart of FIG. 7, the left side of the mathematical formula (4.2) is considered as 2 8 -1. In the description of FIG. 4, when the left side of the mathematical formula (4.2) is set as 2 8i -1 (i = 1, 2, 3, ···) in order to restore the point cloud coordinates more accurately, it was explained that it is necessary to divide the generated RGB signal or YUV signal into i pieces of 8 bits each and create i image files.

[0062] When the value that can be used for the conversion of the point cloud coordinates is 2 8 in this way, when restoring the point cloud coordinates from the image file, after rearranging the color signals extracted from the image file in the order at the time of creating the image file, inverse transformation is performed. For example, when the value that can be used for the conversion of the point cloud coordinates is 2 16 , the image file includes the upper 8 bits (R 上8ビット , G 上8ビット , B 上8ビット ) of the generated color signal and the lower 8 bits (R 下8ビット , G下8ビット , B 下8ビット It consists of two files configured by (). The RGB signals extracted from this image file are (R 16ビット , G 16ビット , B 16ビット ). So that it becomes (), R 上8ビット R 下8ビット , G 上8ビット G 下8ビット , B 上8ビット B 下8ビット are arranged in this order. Then, for each of these RGB signals, the inverse transformation of Equation (4.3) is performed.

[0063] Note that in this embodiment, the description is made with the idea that in the future, the acquired point cloud can be used to reproduce the city in 3D on a PC. If the point clouds of objects not related to communication facilities such as the ground and the wall surfaces of houses are deleted, only the 3D point cloud data of the communication facilities remains. With such 3D point cloud data, when the point cloud is displayed on the screen, it becomes very difficult to recognize the position of the communication facilities. Therefore, the data processing device of the present invention can also hold the point clouds other than the communication facilities. However, depending on the method of using the 3D point cloud data, it may be sufficient to hold only the data of the communication facilities. In that case, the compression rate of the 3D point cloud data can also be improved by deleting the point clouds identified as objects other than the communication facilities in the description of FIG. 2.

[0064] (Embodiment 2) The data processing device 301 can also be realized by a computer and a program, and it is also possible to record the program on a recording medium or provide it through a network. FIG. 8 shows a block diagram of the system 100. The system 100 includes a computer 105 connected to a network 135.

[0065] Network 135 is a data communication network. Network 135 may be a private network or a public network, and may include any or all of (a) a personal area network covering, for example, a certain room, (b) a local area network covering, for example, a certain building, (c) a campus area network covering, for example, a certain campus, (d) a metropolitan area network covering, for example, a certain city, (e) a wide area network covering, for example, an area spanning city, regional, or national boundaries, or (f) the Internet. Communication is performed via network 135 by means of electronic signals and optical signals.

[0066] Computer 105 includes a processor 110 and a memory 115 connected to processor 110. Although computer 105 is represented herein as a stand-alone device, it is not so limited and may rather be connected to other devices not shown in a distributed processing system.

[0067] Processor 110 is an electronic device composed of logic circuits that respond to and execute instructions.

[0068] Memory 115 is a tangible computer-readable storage medium encoded with a computer program. In this regard, memory 115 stores data and instructions, i.e., program code, that are readable and executable by processor 110 to control the operation of processor 110. Memory 115 can be realized by a random access memory (RAM), a hard drive, a read-only memory (ROM), or a combination thereof. One of the components of memory 115 is program module 120.

[0069] Program module 120 includes instructions for controlling processor 110 to execute the processes described herein. Although operations are described herein as being performed by computer 105 or a method or process or a sub-process thereof, those operations are actually performed by processor 110.

[0070] The term "module" is used herein to refer to a functional operation that can be embodied as either a stand-alone component or an integrated configuration consisting of a plurality of sub-components. Thus, program module 120 can be implemented as a single module or as a plurality of modules operating in cooperation with each other. Further, program module 120 is described herein as being installed in memory 115 and thus implemented in software, but it can be implemented in any of hardware (e.g., electronic circuitry), firmware, software, or a combination thereof.

[0071] Program module 120 is shown as already loaded into memory 115, but it may be configured to be located on storage device 140 for later loading into memory 115. Storage device 140 is a tangible computer-readable storage medium that stores program module 120. Examples of storage device 140 include a compact disc, magnetic tape, read-only memory, optical storage medium, a memory unit composed of a hard drive or a plurality of parallel hard drives, and a universal serial bus (USB) flash drive. Alternatively, storage device 140 can be a random access memory or other type of electronic storage device located in a remote storage system (not shown) and connected to computer 105 via network 135.

[0072] System 100 further includes data sources 150A and 150B, which are collectively referred to herein as data source 150 and are communicatively connected to network 135. In practice, data source 150 can include any number of data sources, i.e., one or more data sources. Data source 150 includes unstructured data and can include social media.

[0073] System 100 further includes user device 130, which is operated by user 101 and connected to computer 105 via network 135. Examples of user device 130 include input devices such as a keyboard or a voice recognition subsystem that enable user 101 to convey selections of information and commands to processor 110. User device 130 further includes an output device such as a display device or a printer or a voice synthesizer. A cursor control unit such as a mouse, a trackball, or a touch-sensitive screen enables user 101 to manipulate a cursor on the display device to convey further selections of information and commands to processor 110.

[0074] Processor 110 outputs the result 122 of the execution of program module 120 to user device 130. Alternatively, processor 110 can direct the output to a storage device 125 such as a database or a memory, or to a remote device (not shown) via network 135.

[0075] For example, a program that performs the flowcharts of FIGS. 4 to 7 can be used as program module 120. System 100 can be operated as an image conversion unit 31, an image compression unit 34, and a restoration unit 36, respectively.

[0076] The term "comprising" or "including" is to be construed as specifying the presence of the features, integers, steps, or components recited therein but not precluding the presence of one or more other features, integers, steps, or components, or groups thereof. The terms "a" and "an" are indefinite articles and thus do not preclude embodiments having a plurality thereof.

[0077] (Other embodiments) Note that the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention. In short, the present invention is not limited to the upper embodiments as they are, and at the implementation stage, the components can be modified and embodied without departing from the gist thereof.

[0078] Also, various inventions can be formed by appropriately combining the plurality of components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0079] (Advantages of the invention) What advantages is the point cloud data compression program according to the present disclosure considered to have over the prior art?

[0080] In the prior art, since the entire 3D point cloud data is converted into an image file at once and an image compression technique that is irreversible compression is applied to the image file, the color signal is converted into a different signal during compression, which affects the coordinate accuracy during restoration. In contrast, the data processing apparatus of the present invention can identify facilities from 3D point cloud data and adjust the coordinate accuracy during restoration by changing the compression rate for each facility. Therefore, it is possible to maintain the 3D model creation accuracy while compressing the 3D point cloud data, and it is possible to apply the facility inspection technique using a 3D laser scanner even after compressing the 3D point cloud data.

Description of reference numerals

[0081] 11: 3D Laser Scanner 12: Equipment (Outdoor Structure) 21: Identification Surface 22: Region of Class 1 23: Region of Class 2 24: Identification Object 1 Included in Class 1 25: Identification Object 2 Included in Class 2 30: Object to be Measured Identification Processing Unit 31: Image Conversion Unit 32: Coordinate Identification Unit 33: Image Creation Unit 34: Image Compression Unit 35: Memory Unit 36: Restoration Unit 37: Display Unit 100: System 101: User 105: Computer 110: Processor 115: Memory 120: Program Module 122: Result 125: Storage Device 130: User Device 135: Network 140: Storage Device 150: Data Source 301: Data Processing Device

Claims

1. A data processing apparatus for processing 3D point cloud data representing the 3D coordinates of points on the surface of an outdoor structure obtained by a 3D laser scanner, comprising: an image conversion unit that generates an image file for each outdoor structure, regarding the 3D coordinates of each point in the 3D point cloud data as color signals; an image compression unit that performs image compression processing on each of the image files at a compression rate determined for each type of the outdoor structure corresponding to the image file; a storage unit that stores the compressed image files subjected to the image compression processing in association with the parameters used for the image compression processing; characterized by: the image conversion unit: calculates a divisor of the number of point clouds included in the 3D point cloud data for each outdoor structure; when the number of point clouds is 8 or more and not a prime number, setting the number of point clouds as x, and when the number of point clouds is less than 8 or is a prime number, adding dummy data to the 3D point cloud data for each outdoor structure so as to have a divisor other than 1 and the number of point clouds, and setting the sum of the number of point clouds and the number of dummy data as x, or deleting some points from the 3D point cloud data for each outdoor structure and setting the number obtained by subtracting the number of deleted points from the number of point clouds as x, and creates the image file of pixels that can be represented by two divisors of x. A data processing apparatus characterized by the above.

2. The data processing apparatus according to claim 1, further comprising a restoration unit that uses the compressed image file and the parameters stored in the storage unit to convert the compressed image file into points of 3D coordinates to generate restored point cloud data for each outdoor structure.

3. The data processing apparatus according to claim 1 or 2, wherein the image conversion unit converts the numerical value of the color signal corresponding to the 3D coordinates of one point into a binary number, and creates a plurality of the image files for every 8 bits from the top of the binary number.

4. The image compression unit: The data processing apparatus according to any one of claims 1 to 3, characterized in that the image compression unit collectively performs video compression processing on a plurality of the image files generated from the outdoor structures of the same type of the outdoor structure and existing at different locations to obtain the compressed image files.

5. The data processing device according to claim 4, which cites claim 2, wherein the restoration unit extracts any one of the image files from the compressed image file which is the compressed video, converts it into points of three-dimensional coordinates, and generates restored point cloud data for each outdoor structure.

6. A data processing method for processing three-dimensional point cloud data representing the three-dimensional coordinates of points on the surface of an outdoor structure acquired by a three-dimensional laser scanner, comprising: Performing an image conversion process for each outdoor structure to generate an image file in which the three-dimensional coordinates of each point of the three-dimensional point cloud data are regarded as color signals; Performing an image compression process on each of the image files at a compression rate determined for each type of the outdoor structure corresponding to the image file; Associating and storing the compressed image file subjected to the image compression process and the parameters used for the image compression process; and The image conversion process includes: Calculating a divisor of the number of point clouds included in the three-dimensional point cloud data for each outdoor structure; When the number of point clouds is 8 or more and not a prime number, setting the number of point clouds as x, and when the number of point clouds is less than 8 or a prime number, adding dummy data to the three-dimensional point cloud data for each outdoor structure so as to have a divisor other than 1 and the number of point clouds, and setting the sum of the number of point clouds and the number of dummy data as x, or deleting some points from the three-dimensional point cloud data for each outdoor structure and setting the number obtained by subtracting the number of deleted points from the number of point clouds as x; and Creating the image file of pixels that can be represented by two divisors of x. A data processing method characterized by the above.

7. A program for causing a computer to function as the data processing device according to any one of claims 1 to 5.

Images