A method and system for three-dimensional image splicing of mortise and tenon structure based on neural network

Abstract

The present invention discloses a method and system for splicing three-dimensional images of mortise and tenon structures based on a neural network. The method comprises: after obtaining the original two-dimensional image of the mortise and tenon components to be spliced, the original two-dimensional image of the mortise and tenon components to be spliced ​​is input into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image at any angle in the spliced ​​three-dimensional image. The present invention improves the diversity and rationality of training data through an original labeled image group and an artificial labeled image group, and then performs three-dimensional image splicing through a multi-resolution and multi-angle neural network, which effectively improves the robustness and accuracy of image splicing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for splicing three-dimensional images of mortise and tenon structures based on a neural network. Background Art

[0002] 3D image stitching is a technology that combines images taken from multiple perspectives into a complete scene. It has a wide range of applications in many fields such as virtual reality, medical imaging, remote sensing mapping, etc. Current 3D image stitching involves multiple steps such as image registration, fusion, and rendering to ensure that the final stitched image is visually coherent and natural.

[0003] However, the existing traditional image stitching technology uses the feature points of the overlapping area in the image for identification and registration during the image registration process. It is not suitable for the stitching of mortise and tenon structure images without overlapping areas. In addition, the image output after stitching cannot eliminate the interfaces of the mortise and tenon components, resulting in a high stitching error rate. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method and system for stitching three-dimensional images of mortise and tenon structures based on a neural network. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] A neural network-based mortise and tenon structure three-dimensional image splicing method and system, including image acquisition and network operation.

[0006] Acquire image: acquire the original two-dimensional image of the mortise and tenon component to be spliced;

[0007] Network operation: the original two-dimensional image of the mortise and tenon components to be spliced ​​is input into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image at any angle in the spliced ​​three-dimensional image.

[0008] In a specific embodiment, the training method of the preset neural network model includes:

[0009] Acquire an original marked image group and an artificial marked image group, wherein the original marked image group is obtained by collecting and marking multi-angle images of the mortise and tenon structure assembly product and the multi-angle images of each component in the mortise and tenon structure, and the artificial marked image group is obtained by performing multi-level equal-proportional resolution reduction processing and image preprocessing on the original marked image group;

[0010] The original labeled image group and the artificial labeled image group are input into the neural network model to be trained to obtain the desired neural network model.

[0011] In a specific embodiment, the collecting of the multi-angle images of the finished mortise and tenon structure assembly and the multi-angle images of each component in the mortise and tenon structure includes:

[0012] Multi-angle images of the finished mortise and tenon structure assembly and multi-angle images of each component in the mortise and tenon structure are collected, so that the multi-angle images of the finished mortise and tenon structure assembly and the multi-angle images of each component in the mortise and tenon structure cover all visible surfaces of the corresponding mortise and tenon structure assembly or the corresponding components.

[0013] In a specific embodiment, the marking of the multi-angle images of the finished mortise and tenon structure assembly product and the multi-angle images of each component in the mortise and tenon structure includes: distinguishing marks on the mortise and tenon interfaces in the multi-angle images of the finished mortise and tenon structure assembly product and the multi-angle images of each component in the mortise and tenon structure, wherein the distinguishing marks include distinguishing interface category marks and distinguishing interface shape marks.

[0014] In a specific embodiment, performing multi-level equal-proportional resolution reduction processing and image preprocessing on the original marked image group includes:

[0015] The original marked image is scaled down horizontally and vertically in equal proportions, with each level of reduction being the same and the number of reduction levels being not less than the number of levels input to the neural network;

[0016] The image preprocessing of the original marked image group includes: performing one or more of highlight processing, color processing, occlusion processing and noise processing operations on the original marked image group.

[0017] In one embodiment, the neural network comprises:

[0018] A plurality of parallel topology processing units and a fusion processing unit;

[0019] Each parallel topology processing unit includes multiple serial subunits, a first fully connected layer and a first activation function, wherein the first activation function is obtained by fusing a diffusion exponential linear function and a linear piecewise function, and the fusion parameter is determined by the resolution;

[0020] The fusion processing unit includes multiple parallel sub-units, a second activation function and a second fully connected layer, wherein the second activation function is an 8-point Bessel function.

[0021] In a specific embodiment, the number of the parallel topology processing units is not less than the product of the number of angle graphs and the number of resolution levels;

[0022] When the input image resolutions of the parallel topology processing units are the same, the calculation results of the same-level serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes calculating the mean, calculating the median or Gaussian filtering;

[0023] When the input image resolutions of the parallel topology processing units are different, the calculation results of the penultimate serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes Gaussian filtering;

[0024] Each serial subunit includes 3-5 convolutional layers, 1 2*2 pooling layer, 1 third activation function and 1 2*2 upsampling in sequence, where the third activation function is obtained by fusing a linear rectification function and an exponential linear piecewise function, and the fusion parameter is determined by the resolution;

[0025] The number of serial sub-units in a parallel topology processing unit is determined by the resolution of the input image of the parallel topology processing unit.

[0026] In a specific embodiment, the parallel subunits in the fusion processing unit sequentially include 4-6 convolutional layers, 1 2*2 pooling layer, 1 fourth activation function, 1 2*2 upsampling and 1 third fully connected layer, wherein the fourth activation function is a 12-point Bessel function;

[0027] In a specific embodiment, training the neural network model to be trained includes:

[0028] Set activation function parameters, number and size of convolutional layers;

[0029] Using incremental data input, the original marked image group is input into the neural network model to be trained to obtain first parameters;

[0030] Using incremental data input, after locking the parameters of all convolutional layers in the first parameters, the artificially marked image group is input into the neural network model to be trained to obtain second parameters;

[0031] After adjusting the activation function parameters, the number and size of convolutional layers, multiple second parameters are iterated to obtain a set of second parameters with the highest accuracy, and a set of second parameters with the highest accuracy is selected as the parameters of the preset neural network model.

[0032] The present invention also provides a neural network-based mortise and tenon structure three-dimensional image splicing system, comprising:

[0033] Data acquisition module: used to obtain two-dimensional images of the mortise and tenon components to be spliced;

[0034] Neural network processing module: used to input the two-dimensional image of the mortise and tenon components to be spliced ​​into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image of any angle in the spliced ​​three-dimensional image.

[0035] Beneficial effects of the present invention:

[0036] The present invention provides a neural network-based three-dimensional image stitching method and system for mortise and tenon structures, which realizes three-dimensional image stitching of mortise and tenon structures by using an original labeled image group and an artificial labeled image group and adopting a multi-resolution and multi-angle neural network for three-dimensional image stitching, thereby effectively improving the image stitching robustness and stitching accuracy.

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of a method for stitching three-dimensional images of mortise and tenon structures based on a neural network provided by an embodiment of the present invention;

[0039] Figure 2 It is a block diagram of a method for stitching three-dimensional images of mortise and tenon structures based on a neural network provided by an embodiment of the present invention;

[0040] Figure 3 It is the shooting angle intention in a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a convolutional neural network in a three-dimensional image stitching method of a mortise and tenon structure based on a neural network provided in an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of a serial subunit in a parallel topology processing unit of a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention;

[0043] Figure 6 It is a same-resolution input parallel topology processing unit of a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention;

[0044] Figure 7 A parallel topological processing unit for inputting different resolutions of a three-dimensional image stitching method of a mortise and tenon structure based on a neural network provided by an embodiment of the present invention;

[0045] Figure 8 It is a flow chart of a fusion processing unit of a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention;

[0046] Fig. 9It is a schematic diagram of parallel sub-units in a fusion processing unit of a neural network-based mortise and tenon structure three-dimensional image stitching method provided in an embodiment of the present invention; DETAILED DESCRIPTION

[0047] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0048] Embodiment 1

[0049] In one specific embodiment, see Figure 1 , Figure 1 This is a flowchart of a three-dimensional image stitching method for mortise and tenon structures based on a neural network.

[0050] S1: Acquire an original two-dimensional image of the mortise and tenon assembly to be spliced, wherein the acquisition method of the original two-dimensional image includes:

[0051] Camera shooting: The shooting equipment includes any device with a visible light camera, including professional cameras, sports cameras, mobile phones and camera drones;

[0052] Machine scanning: Scanning equipment includes any device with a light source and scanning function, including scanners;

[0053] Infrared acquisition; the shooting equipment includes any device with an infrared camera, including infrared cameras and;

[0054] The original two-dimensional image includes digital photos and original raw data in any format, wherein the formats of digital photos include bmp images, raw images, jpg images and png images, and the original raw data includes YUV data, RGB data and grayscale data.

[0055] S2: Inputting the original two-dimensional image of the mortise and tenon assembly to be spliced ​​into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image of any angle in the spliced ​​three-dimensional image, wherein the three-dimensional image is saved in the following manner:

[0056] 3D images are saved in the form of point clouds, which are data sets consisting of a large number of discrete points, each of which contains its coordinates (X, Y, Z) in three-dimensional space and possible other information (such as color, intensity, etc.);

[0057] 3D images are stored as 2D images with depth information, which refers to the representation of the distance between the object in the image and the observer. This information can be obtained by various means, such as stereo vision, time-of-flight (ToF) cameras, structured light scanning, etc.

[0058] Three-dimensional images can be saved in computers through volume rendering technology, which converts three-dimensional data sets (such as CT or MRI scan data in medical imaging) into visual images;

[0059] The three-dimensional image can be obtained by software capture and image screenshot to obtain a two-dimensional image at any angle. Since the processing and storage of three-dimensional images usually require more computing resources than two-dimensional images, multiple two-dimensional images are usually used for storage.

[0060] Since VR / AR technology can provide a highly immersive experience and users can interact with virtual objects through various interactive methods, three-dimensional images are displayed using VR / AR.

[0061] For VR: A head-mounted display provides users with a completely virtual environment in which they can move and explore freely, and experience the immersive experience brought by the three-dimensional mortise and tenon images.

[0062] For AR: Overlaying 3D images onto the user’s real world, enabled by devices such as smartphones, tablets, or AR glasses. AR displays allow users to interact with 3D mortise and tenon images while maintaining awareness of the real environment.

[0063] Due to the problems of low image accuracy, poor effect, color deviation, brightness difference, etc. in actual applications, in order to increase the robustness of the neural network and improve the accuracy of stitching 3D images, it is necessary not only to shoot a large number of original labeled image groups, but also to add a large number of artificial labeled image groups to improve the richness of the training set. In a specific embodiment, the entire process of making the original labeled image group and the artificial labeled image group can be found in Figure 2 , Figure 2 The present invention is a block diagram of a 3D image stitching method of a mortise and tenon structure based on a neural network, and the specific steps include:

[0064] S21: photographing the multi-angle images of the finished product of the mortise and tenon structure and the multi-angle images of each component in the mortise and tenon structure to obtain an original image group. Figure 3 , Figure 3It is the shooting angle intention in a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention. The shooting angle can be symmetrical shooting, multi-angle bisection shooting, and multi-angle coverage shooting; the shooting environment includes daylight, indoors, and night scenes; the shooting angle and distance can be shot at different angles at multiple distances; the shooting clarity must be able to see the position and shape of the mortise and tenon interface. The mortise and tenon structure is characterized by being able to achieve a firm connection through the concave and convex fit between the wood without the use of nails or glue. This structure not only played an important role in ancient architecture and furniture, but also remains valuable in modern design and industrial fields. There are many types of mortise and tenon structures, including but not limited to the following:

[0065] Dovetail: The end of the tenon is shaped like a swallow's tail and is usually used to enhance the stability of the connection.

[0066] Tenon and mortise: commonly used in table and desk furniture, with the top of the leg connected to the table top with a tenon, and the tooth bar embedded between the leg and the tooth.

[0067] Zongjiao tenon: It is named because its shape resembles the corner of a zongzi, and is mostly used to connect frame structures.

[0068] Wedge tenon: used to connect round or curved furniture parts, such as round armrests.

[0069] Overlord stretcher: used for furniture such as square tables and square stools. It does not use horizontal stretchers to reinforce the legs, but instead uses an S-shaped mortise and tenon structure to achieve connection.

[0070] Shoulder tenon: There is a half-straight tenon on the top of the leg, which is connected to the mortise on the large edge of the table top to form a stable support.

[0071] S22: Performing distinguishing marks on the original image group to obtain an original marked image group, wherein the distinguishing marks include distinguishing interface category marks and distinguishing interface shape marks. The basic rules for distinguishing marks include:

[0072] 1. The mortise and tenon joints are marked as consistent;

[0073] 2. If there are multiple mortise and tenon components with a certain mortise and tenon sequence, distinguish the mortise and tenon sequence marks;

[0074] 3. If there are multiple mortise and tenon structures or mortise and tenon structures with the same interfaces, mark the locations to distinguish them.

[0075] S23: Perform multi-level equal-proportional resolution reduction processing and image preprocessing on the original marked image group to obtain an artificial marked image group. In a specific implementation, multi-level equal-proportional resolution reduction processing can be performed first, and then image preprocessing can be performed.

[0076] S231: The multi-level equal-proportional resolution reduction process includes:

[0077] The marked image is scaled down horizontally and vertically by 2:1 in area, and the ratio of multiple levels of reduction remains the same; the number of reduction levels is equal to the number of neural network input levels. The reduction method must ensure that there is no distortion and no jagged edges after reduction. The following reduction processing methods can be used:

[0078] Bilinear interpolation: This method calculates the target pixel value by taking a weighted average of the four nearest neighbor pixels, resulting in a smoother image. It provides a compromise between quality and computational speed.

[0079] Bicubic interpolation: This is a more complex interpolation method that calculates the target pixel value by taking a weighted average of 16 neighboring pixels. It can produce very smooth images, but is more computationally expensive.

[0080] Lanczos resampling: This method uses a sine window function to calculate the interpolation, which can retain more high-frequency details and is suitable for high-quality image scaling.

[0081] Wavelet transform: Wavelet transform is a multi-resolution analysis tool that can decompose and reconstruct images at different scales, thereby achieving high-quality image reduction.

[0082] Deep learning methods: In recent years, deep learning techniques, especially convolutional neural networks (CNNs), have been used for image super-resolution and scaling tasks. These methods can optimize the scaling process by learning from large amounts of image data.

[0083] Adaptive Adjustment: Some advanced methods dynamically adjust the interpolation strategy during scaling according to specific features of the image content (e.g., edges, textures) to maintain image quality.

[0084] S232: In a specific implementation, taking a typical design as an example, the image preprocessing includes:

[0085] Three types of compensation brightness processing are distinguished, namely central brightness compensation, global brightness compensation and surrounding brightness compensation;

[0086] Distinguish 4 types of noise morphology processing, namely frequency domain denoising, time domain denoising, salt and pepper noise compensation and white noise compensation;

[0087] Distinguish 4 types of color transformation processing, specifically saturation enhancement processing, protection color enhancement processing, grayscale processing and color inversion processing;

[0088] The above-mentioned processing is cross-combined to form a multi-process processing. A typical example of a multi-process processing is to first perform central brightness compensation, then perform time domain denoising, and finally enhance saturation.

[0089] In order to ensure the accuracy and robustness of the neural network and to be compatible with the input of images of different resolutions and angles, not only a parallel processing network is required for image preprocessing, but also a fusion processing network is required for image stitching. Figure 4 , Figure 4 It is a schematic diagram of a convolutional neural network in a neural network-based mortise and tenon structure three-dimensional image stitching method provided in an embodiment of the present invention, including multiple parallel topology processing units and a fusion processing unit.

[0090] In a specific embodiment, the topological parallel processing unit is described in Figure 5 , Figure 5 This is a schematic diagram of serial subunits in a parallel topology processing unit of a neural network-based mortise and tenon structure three-dimensional image stitching method provided by an embodiment of the present invention, each parallel topology processing unit includes multiple serial subunits, a first fully connected layer and a first activation function. The first activation function is obtained by fusing a diffusion exponential linear function and a linear piecewise function.

[0091] The fusion parameters are determined by the resolution. If the input image resolution exceeds the threshold 1, the first activation function is the diffusion exponential linear function; if the input image resolution is less than the threshold 2, the first activation function is a linear piecewise function; otherwise, the larger the input image resolution, the greater the fusion proportion of the diffusion exponential linear function.

[0092] The diffusion index linear function is shown in Equation 1, where a, b, and c need to be set manually.

[0093]

[0094] The linear piecewise function is shown in Equation 2, where a and b need to be set manually.

[0095]

[0096] In order to improve the accuracy of the neural network output, a serial processing unit is required based on comprehensive considerations of resolution and angle, while improving the characteristic differences of feature maps at different angles and eliminating invalid feature maps, ultimately improving the robustness of the neural network's feature matching. The specific solution is:

[0097] If the input image resolution is the same but the shooting angles are different, the output feature maps of different angles are averaged and stabilized through cross-fusion between serial sub-units at the same level to ensure the uniqueness of the feature maps output by the parallel topology processing unit and improve the feature difference;

[0098] If the input images have the same resolution and shooting angles, the serial sub-units at the same level will cross-fuse to eliminate the characteristic differences of the output feature maps with basically the same angles, so as to ensure that there are no invalid feature maps, thereby ensuring the accuracy of the subsequent feature matching fusion;

[0099] If the input images are shot at the same angle and have different resolutions, the serial subunit at the front stage is used to adjust and align the image size, and the serial subunit at the back stage is used to average the feature maps after image size alignment to increase the precision and accuracy of feature maps at the same angle, thereby ensuring the accuracy of feature matching and fusion in the later stage;

[0100] The application method of this scheme in a specific neural network is:

[0101] See also Figure 6 , Figure 6 The invention provides a parallel topological processing unit with the same resolution input in a mortise and tenon structure three-dimensional image stitching method based on a neural network. When the input image resolutions of the parallel topological processing units are the same, the calculation results of the serial subunits at the same level in each parallel topological processing unit are cross-fused, wherein the cross-fusion method includes calculating the mean, calculating the median or Gaussian filtering;

[0102] See also Figure 7 , Figure 7 The present invention provides a method for stitching three-dimensional images of mortise and tenon structures based on a neural network, wherein the input images of the parallel topology processing units have different resolutions, and when the input images of the parallel topology processing units have different resolutions, the calculation results of the penultimate serial subunits in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes Gaussian filtering;

[0103] Take a typical design as an example: each serial subunit includes 5 10*10 convolution layers, 1 2*2 pooling layer, 1 third activation function and 1 2*2 upsampling in sequence, where the third activation function is an 8-point Bessel function; and each 10*10 convolution layer is quantized to 8 bits.

[0104] Input image resolution is large, medium and small:

[0105] For large-resolution images, the number of serial subunits in the topological parallel processing unit is 16;

[0106] For medium-resolution images, the number of serial subunits in the topological parallel processing unit is 8;

[0107] For small resolution images, the number of serial sub-units in the topological parallel processing unit is four.

[0108] In a specific embodiment, the fusion processing unit is described in Figure 8 , Figure 8 The present invention provides a fusion processing unit flow diagram of a neural network-based mortise and tenon structure three-dimensional image stitching method, wherein the fusion processing unit includes multiple parallel subunits, a second activation function, and a second fully connected layer. The second activation function is obtained by fusing a linear rectification function and an exponential linear piecewise function.

[0109] The fusion parameters are determined by the resolution. If the input image resolution exceeds the threshold 1, the first activation function is the diffusion exponential linear function; if the input image resolution is less than the threshold 2, the first activation function is a linear piecewise function; otherwise, the larger the input image resolution, the greater the fusion proportion of the diffusion exponential linear function.

[0110] The linear rectification function is shown in Equation 3, where a and b need to be set manually.

[0111] F(x)=max(0,ax+b) (3)

[0112] The exponential linear piecewise function is shown in formula 4, where a, b, c, d, e need to be set manually, and k1 and k2 need to be set manually

[0113]

[0114] Take a typical design as an example:

[0115] See also Fig. 9 , Fig. 9 It is a schematic diagram of parallel sub-units in a fusion processing unit of a mortise and tenon structure three-dimensional image stitching method based on a neural network provided in an embodiment of the present invention. Each parallel sub-processing unit includes, in sequence, 6 8*8 convolutional layers, 1 2*2 pooling layer, 1 fourth activation function, 1 2*2 upsampling and 1 third fully connected layer, wherein the fourth activation function is a 12-point Bessel function.

[0116] After the neural network is determined, it needs to be trained to obtain all the parameters of the neural network. In a specific embodiment, the neural network training method is as follows:

[0117] S24: Set activation function parameters, number and size of convolutional layers;

[0118] S25: using incremental data input, inputting the original labeled image group into the model for training, and obtaining the first parameter after multiple iterations convergence, wherein the iteration process is to adjust the number and size of convolutional layers;

[0119] S26: using incremental data input, locking the parameters of all convolutional layers in the first parameter, inputting the artificially marked image group into the model for training, and obtaining the second parameter after multiple iterations converge, wherein the iteration process is to adjust the size of the convolutional layer;

[0120] After adjusting the activation function parameters, the number and size of convolutional layers, and repeating steps S24-S26 to obtain multiple second parameters, a set of second parameters with the highest accuracy is selected as the preset neural network parameters. In practical applications, the accuracy of image stitching can be verified by comparing the difference between the stitched image and the real scene, or evaluated by the user's visual perception. For example, high-quality image stitching should have no visually obvious stitching marks, such as cracks, dislocations, or unnatural transitions.

[0121] In order to improve the training accuracy, the training process uses the weighted loss function of structural similarity SSIM and mean square error MSE; during the adjustment process, if you plan to increase the training speed, directly use mean square error MSE as the loss function.

[0122] The present invention also provides a neural network-based mortise and tenon structure three-dimensional image splicing system, comprising:

[0123] Data acquisition module: obtains the two-dimensional image of the mortise and tenon components to be spliced;

[0124] Neural network processing module: input the two-dimensional image of the mortise and tenon components to be spliced ​​into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image of any angle in the spliced ​​three-dimensional image.

[0125] The present invention provides a neural network-based three-dimensional image stitching method and system for mortise and tenon structures, which realizes three-dimensional image stitching of mortise and tenon structures by using an original labeled image group and an artificial labeled image group and adopting a multi-resolution and multi-angle neural network for three-dimensional image stitching, thereby effectively improving the image stitching robustness and stitching accuracy.

[0126] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0127] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps.

[0128] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A three-dimensional image stitching method of mortise and tenon structure based on neural network, characterized in that: include: Acquire the original two-dimensional image of the mortise and tenon assembly to be spliced; Inputting the original two-dimensional image of the mortise and tenon joint assembly to be spliced ​​into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image at any angle in the spliced ​​three-dimensional image; The neural network comprises: A plurality of parallel topology processing units and a fusion processing unit; Each parallel topology processing unit includes multiple serial subunits, a first fully connected layer and a first activation function, wherein the first activation function is obtained by fusing a diffusion exponential linear function and a linear piecewise function, and the fusion parameter is determined by the resolution; The fusion processing unit includes a plurality of parallel sub-units, a second activation function and a second fully connected layer, wherein the second activation function is an 8-point Bessel function; The number of parallel topological processing units is not less than the product of the number of angle images and the number of resolution levels; When the input image resolutions of the parallel topology processing units are the same, the calculation results of the same-level serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes calculating the mean, calculating the median or Gaussian filtering; When the input image resolutions of the parallel topology processing units are different, the calculation results of the penultimate serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes Gaussian filtering; Each serial subunit includes 3-5 convolutional layers, 1 2*2 pooling layer, 1 third activation function and 1 2*2 upsampling in sequence, where the third activation function is obtained by fusing a linear rectification function and an exponential linear piecewise function, and the fusion parameter is determined by the resolution; The number of serial sub-units in a parallel topology processing unit is determined by the resolution of the input image of the parallel topology processing unit.

2. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 1, characterized in that: The training method of the preset neural network model includes: Acquire an original marked image group and an artificial marked image group, wherein the original marked image group is obtained by collecting and marking multi-angle images of the mortise and tenon structure assembly product and the multi-angle images of each component in the mortise and tenon structure, and the artificial marked image group is obtained by performing multi-level equal-proportional resolution reduction processing and image preprocessing on the original marked image group; The original labeled image group and the artificial labeled image group are input into the neural network model to be trained to obtain the desired neural network model.

3. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 2, characterized in that: Collecting multi-angle images of the finished product of the mortise and tenon structure and multi-angle images of each component in the mortise and tenon structure, including: Multi-angle images of the finished mortise and tenon structure assembly and multi-angle images of each component in the mortise and tenon structure are collected, so that the multi-angle images of the finished mortise and tenon structure assembly and the multi-angle images of each component in the mortise and tenon structure cover all visible surfaces of the corresponding mortise and tenon structure assembly or the corresponding components.

4. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 2, characterized in that: The multi-angle images of the finished mortise and tenon structure assembly and the multi-angle images of each component in the mortise and tenon structure are marked, including: The mortise and tenon interfaces in the multi-angle images of the finished mortise and tenon structure assembly and the multi-angle images of each component in the mortise and tenon structure are distinguished by markings, wherein the distinguishing marks include marks for distinguishing interface categories and marks for distinguishing interface shapes.

5. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 2, characterized in that: The multi-level equal-proportional resolution reduction processing and image preprocessing of the original marked image group include: The original marked image is scaled down horizontally and vertically in equal proportions, with each level of reduction being the same and the number of reduction levels being not less than the number of levels input to the neural network; The image preprocessing of the original marked image group includes: performing one or more of highlight processing, color processing, occlusion processing and noise processing operations on the original marked image group.

6. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 1, characterized in that: The parallel subunits in the fusion processing unit include 4-6 convolutional layers, 1 2*2 pooling layer, 1 fourth activation function, 1 2*2 upsampling and 1 third fully connected layer in sequence, wherein the fourth activation function is a 12-point Bessel function.

7. The neural network-based mortise and tenon structure three-dimensional image stitching method according to claim 2, characterized in that: Training the neural network model to be trained includes: Set activation function parameters, number and size of convolutional layers; Using incremental data input, the original marked image group is input into the neural network model to be trained to obtain first parameters; Using incremental data input, after locking the parameters of all convolutional layers in the first parameters, the artificially marked image group is input into the neural network model to be trained to obtain second parameters; After adjusting the activation function parameters, the number and size of convolutional layers, multiple second parameters are iterated to obtain a set of second parameters with the highest accuracy, and a set of second parameters with the highest accuracy is selected as the parameters of the preset neural network model.

8. A three-dimensional image stitching system of mortise and tenon structure based on neural network, characterized by: include: A data acquisition module, used to obtain a two-dimensional image of the mortise and tenon components to be spliced; A neural network processing module, used for inputting the two-dimensional image of the mortise and tenon components to be spliced ​​into a preset neural network model to obtain a spliced ​​three-dimensional image or a two-dimensional image at any angle in the spliced ​​three-dimensional image; The neural network comprises: A plurality of parallel topology processing units and a fusion processing unit; Each parallel topology processing unit includes multiple serial subunits, a first fully connected layer and a first activation function, wherein the first activation function is obtained by fusing a diffusion exponential linear function and a linear piecewise function, and the fusion parameter is determined by the resolution; The fusion processing unit includes a plurality of parallel sub-units, a second activation function and a second fully connected layer, wherein the second activation function is an 8-point Bessel function; The number of parallel topological processing units is not less than the product of the number of angle images and the number of resolution levels; When the input image resolutions of the parallel topology processing units are the same, the calculation results of the same-level serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes calculating the mean, calculating the median or Gaussian filtering; When the input image resolutions of the parallel topology processing units are different, the calculation results of the penultimate serial sub-units in each parallel topology processing unit are cross-fused, wherein the cross-fusion method includes Gaussian filtering; Each serial subunit includes 3-5 convolutional layers, 1 2*2 pooling layer, 1 third activation function and 1 2*2 upsampling in sequence, where the third activation function is obtained by fusing a linear rectification function and an exponential linear piecewise function, and the fusion parameter is determined by the resolution; The number of serial sub-units in a parallel topology processing unit is determined by the resolution of the input image of the parallel topology processing unit.

Images