Multi-view picture data acquisition method based on deep learning

Abstract

The invention discloses a multi-view picture data acquisition method based on deep learning, and the method comprises the following steps: S1, collecting picture data from different views at the same time through a picture collection device, and generating a multi-view original picture; s2, carrying out preprocessing on the collected multi-view-angle original picture; s3, utilizing an improved ResNet34 network to perform feature extraction on the preprocessed pictures, optimizing a feature extraction process through an attention mechanism, and generating feature maps of the pictures of different viewing angles; s4, performing multi-level fusion on the feature maps of the different view angle pictures, and introducing picture space information to generate comprehensive features of the multi-view angle pictures; and S5, based on the comprehensive features of the multi-view image, using the improved generative adversarial network to carry out image reconstruction, and outputting finally processed image data to complete image data acquisition. According to the method, the improved deep learning method is utilized, picture reconstruction is carried out through multi-view image feature fusion and the generative adversarial network, and the method has the advantages of high precision, high efficiency and high adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of image data acquisition, and in particular to a multi-view image data acquisition method based on deep learning. Background Art

[0002] In the field of computer vision, the rapid development of image acquisition and processing technology has made the acquisition of multi-view image data a crucial research direction. Traditional multi-view image acquisition technology mainly relies on capturing image information of scenes at different perspectives through multiple static cameras or image acquisition devices. By combining images collected from multiple perspectives with image reconstruction technology in computer vision, three-dimensional images, models or scenes can be generated, which are widely used in 3D reconstruction, virtual reality, augmented reality, robot navigation, autonomous driving, medical imaging and other fields. However, the existing technology still has some limitations in multi-view image acquisition, image reconstruction and optimization, which need to be improved urgently.

[0003] At present, traditional multi-view image data acquisition methods mostly capture image data through camera arrays or manually setting the camera position. These methods usually rely on manually setting the position and viewing angle of the acquisition equipment, which makes the acquisition process cumbersome and difficult to optimize. In addition, the camera's angle, focal length and other parameters are often statically set, lacking intelligent adjustment and optimization methods. In terms of preprocessing of multi-view images, most existing technologies rely on traditional image denoising, geometric correction and color standardization methods. Although these methods can improve image quality to a certain extent, they often cannot comprehensively extract and optimize deep-level features in the image. Especially for multi-view images in complex scenes, existing preprocessing methods often have problems with information loss and insufficient detail retention, and cannot maximize the extraction of key information in the image.

[0004] In terms of image feature extraction, most existing technologies rely on traditional convolutional neural networks for feature learning. Although convolutional neural networks have achieved remarkable results in tasks such as image classification and target detection, they have certain limitations in the data processing of multi-view images. In particular, in the feature extraction process of multi-view images, traditional convolutional neural networks often cannot effectively process the image differences between different viewpoints, resulting in incomplete extracted feature information and inability to accurately capture the spatial relationship and depth information in multi-view images. In addition, traditional convolutional neural networks usually lack an adaptive feature selection mechanism, and are prone to ignoring the details of certain important areas in the image, which cannot meet the needs of high-precision image reconstruction.

[0005] In order to solve these problems, in recent years, deep learning-based generative adversarial networks have been widely used in image reconstruction and generation tasks. Generative adversarial networks can generate high-quality images under unsupervised conditions through adversarial training of generators and discriminators. However, when processing multi-view images, existing generative adversarial networks often cannot effectively fuse image features from different viewpoints, resulting in poor reconstruction effects. Especially in the three-dimensional reconstruction process of multi-view images, traditional generative adversarial networks usually lack the ability to process spatial information, resulting in obvious distortion in the spatial structure of the generated images. Therefore, how to effectively combine the feature information of multi-view images and optimize the image reconstruction process is still a technical difficulty.

[0006] In addition, existing image reconstruction methods often rely on a large amount of manually annotated data during the training process of the generative adversarial network, and the training time is long and the computing resources are high. Traditional generative adversarial network models require a lot of adversarial training between the generator and the discriminator, which usually leads to unstable training and difficult convergence. Moreover, due to the lack of effective supervision signals during the training process, the network may experience problems such as mode collapse, which affects the quality of image generation and the accuracy of multi-view image reconstruction. Summary of the invention

[0007] One purpose of the present invention is to propose a multi-view image data acquisition method based on deep learning. The present invention extracts features from multi-view images through an improved ResNet34 network and uses an attention mechanism to weight the features, thereby optimizing the problems of insufficient feature selection and image detail capture in traditional networks in multi-view image processing. The improved generative adversarial network is applied to image reconstruction, and images are generated using optimized generator and discriminator structures. The details and spatial structure of the images are optimized through a feedback mechanism during training.

[0008] A multi-view image data acquisition method based on deep learning according to an embodiment of the present invention includes the following steps: S1. Collect image data from different perspectives simultaneously through an image acquisition device to generate multi-perspective original images; S2, preprocessing the collected multi-view original images; S3. Use the improved ResNet34 network to extract features from the preprocessed images, and optimize the feature extraction process through the attention mechanism to generate feature maps of images from different perspectives; S4, perform multi-level fusion of feature maps of images from different perspectives, introduce image spatial information, and generate comprehensive features of multi-perspective images; S5. Based on the comprehensive features of multi-view images, the improved generative adversarial network is used to reconstruct the image, output the final processed image data, and complete the image data collection.

[0009] Optionally, the preprocessing includes denoising, geometric correction, color standardization and image alignment.

[0010] Optionally, the S3 specifically includes: S31, constructing an improved ResNet34 network, wherein the improved ResNet34 network includes: Input layer, input preprocessed image data; The convolution layer performs multi-scale feature extraction on the input image data at multiple scales to capture the local and global features of the image data. The convolution kernel sizes used in the convolution layer are 7×7 and 3×3. Batch normalization layer, which normalizes the output of each layer; The activation function layer performs nonlinear activation on the output of the convolutional layer through the ReLU activation function; The residual block directly adds the input to the output layer through a skip connection to keep the information flow smooth. The improved ResNet34 network introduces a residual connection with variable weights in the residual block based on the traditional ResNet34 network; Pooling layer, which applies max pooling after convolution, selects the maximum value in each 2×2 window and reduces the size of the feature map; The fully connected layer maps the features to the target output space by performing a weighted summation of the outputs of the previous layers and applying an activation function; Output layer, generating the final result of the improved ResNet34 network; S32, input the preprocessed image into the improved ResNet34 network for feature extraction, each residual block of the improved ResNet34 network includes two convolution layers, each convolution layer is followed by batch normalization and ReLU activation function, and the output is obtained through the convolution operation, and the output is fused with the input through the addition operation: ; ; in, represents the kth feature map extracted by improving the ResNet34 network, represents the weight of the kth convolution kernel, represents the kth bias term, represents the input image, (i, j) represents the pixel coordinates of each position in the image, m and n represent the size of the convolution kernel, represents the feature map extracted by the convolutional layer, represents the adaptive parameters; S33. Based on the attention mechanism, an attention weight vector is generated for each feature map through a fully connected layer, and the feature map is weighted to enhance the key information in the feature map and generate feature maps of pictures from different perspectives. The attention mechanism selects the degree of attention of the picture area in an adaptive manner: ; ; in, represents the attention weight vector of the kth feature map, σ represents the activation function, represents the weight of the fully connected layer, represents the bias of the fully connected layer, γ represents the adaptive weighting coefficient, Represents the weighted feature map.

[0011] Optionally, the S4 specifically includes: S41. Fuse the feature maps of images from different perspectives: ; in, represents the fused features, Representation perspective The corresponding weight coefficient is obtained through the training process, where N represents the number of viewing angles; S42, by adding spatial information to improve the perception of the spatial structure of the image features, after the feature fusion, the spatial information and the fused features are connected to obtain the spatial features of the fused spatial information: ; in, represents the spatial features of fused spatial information, represents the feature connection operation, S(x,y) represents the spatial position information of each pixel in the image; S43. Combine multi-view image information, improve attention mechanism and spatial information, and finally generate comprehensive features of multi-view images: ; in, Represents the comprehensive features of multi-view images, Represents the spatial features of the fused spatial information.

[0012] Optionally, the S5 specifically includes: S51, constructing an improved generative adversarial network, wherein the improved generative adversarial network comprises: Convolutional layer, extracts local image features from the comprehensive features of multi-view images, captures low-level image information, and maps the input high-dimensional features to low-dimensional feature maps; Batch normalization layer, normalizes the output of the intermediate layer; Attention mechanism layer, which performs weighted processing on key areas in the image; The decoding layer maps the features optimized by the convolution layer and the attention mechanism layer back to the high-dimensional image space. The decoding layer restores the feature map to the final generated image through deconvolution operations, and generates the image through upsampling and transposed convolution. Generator output layer, outputs the generated image; The discriminator is responsible for determining whether the image generated by the generator is close to the real image, comparing the difference between the real image and the generated image, and outputting a true or false score; S52, receiving the comprehensive features of the multi-view image, and inputting the comprehensive features of the multi-view image into the convolutional layer of the improved generative adversarial network: ; in, Indicates The output feature map of the layer, Indicates The weight matrix of the convolution kernel of the layer, Indicates The bias vector of the layer convolution, Indicates The output feature map of the layer, when When Represents the comprehensive features of the input multi-view image , Conv2D represents a two-dimensional convolution operation, Represents the comprehensive features of the input multi-view images; S53, based on output feature map Generate query and key matrices: ; Where Q represents the query matrix, K represents the key matrix, Indicates The output feature map of the layer, and represents the weight matrix; S54, by calculating the dot product of the query matrix and the key matrix, the attention weight matrix is ​​obtained, normalized, and the attention weight matrix A is combined with the input feature map Combined, we get the weighted output feature map: ; ; Among them, A represents the attention weight matrix, represents the transpose of the key matrix, D represents the dimension of query and key, Softmax represents normalization, represents the weighted output feature map, V represents the value matrix, The weight matrix representing the value matrix; S55. The feature map obtained from the attention mechanism is decoded into a picture output through a decoding layer, wherein the decoding layer uses upsampling and convolution operations and includes two stages: In the first stage, the deconvolution layer is used to upsample the input feature map and restore the spatial size of the image: ; in, represents the restored image, DeConv represents the deconvolution operation, and Represent the weight matrix and bias term of deconvolution respectively; In the second stage, the restored image is processed through the activation function and the decoded image is output: ; Among them, Y represents the processed image, and tanh represents the nonlinear activation function; S56. Input the decoded image into a discriminator, wherein the discriminator includes a convolutional layer, an activation function, and a fully connected layer. The output of the discriminator is a value between 0 and 1: ; Where D(X) represents the output of the discriminator; S57. Use cross entropy loss to construct the loss function of the discriminator: ; Among them, D(X) represents the output of the discriminator for the real picture, D(Y) represents the output of the discriminator for the generated picture, X represents the real picture, and Y represents the generated picture. represents the data distribution, represents the generated distribution, represents the expectation of the data distribution, represents the expectation of the generated distribution, represents the loss function of the discriminator; S58. Use back propagation and gradient descent to update the parameters of the generator and discriminator, minimize the difference between the real image and the generated image, and finally output the generated image data.

[0013] The beneficial effects of the present invention are: First, in the process of feature extraction and fusion, the present invention effectively overcomes the shortcomings of traditional convolutional neural networks in multi-view image processing by introducing an improved ResNet34 network and combining the attention mechanism to optimize the feature extraction process. Traditional convolutional neural networks usually do not fully capture the differences between different perspectives during feature extraction, resulting in an incomplete expression of image features. By introducing a variable weight residual connection in each residual block through the improved ResNet34 network, and combining the attention mechanism to weight the key areas, the features of images from different perspectives can be more accurately extracted and fused. This method not only improves the accuracy of feature extraction, but also optimizes the spatial information perception ability of multi-view images, effectively improving the quality of image reconstruction.

[0014] Secondly, in the image reconstruction stage, the present invention uses an improved generative adversarial network to perform three-dimensional image reconstruction, which has higher stability and stronger image generation capabilities than the traditional generative adversarial network method. In the image generation process, the existing GAN model often has unstable quality of generated images or even obvious distortion due to training instability or mode collapse problems. By improving the structure of the generative adversarial network and combining the comprehensive features of multi-view images for training, the quality and details of the generated images can be effectively improved, ensuring that the final output image is more realistic and clear. In addition, the optimized generator maps the feature map of the multi-view image back to the high-dimensional space through operations such as upsampling and transposed convolution, and further improves the image details through adversarial training of the discriminator, making the image generation more refined and in line with actual needs.

[0015] Finally, the present invention optimizes the image feature extraction and reconstruction process through deep learning, which improves image quality while greatly reducing the need for manual intervention. The entire process not only improves the accuracy and efficiency of image acquisition, but also enhances the reliability and diversity of image data, making the generated images more comprehensive and accurate in reflecting the actual scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 A flowchart of a multi-view image data acquisition method based on deep learning proposed by the present invention; Figure 2 This is a flowchart of the steps of improving the generative adversarial network of the multi-view image data acquisition method based on deep learning proposed in the present invention. DETAILED DESCRIPTION

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0018] refer to Figure 1 and Figure 2 , a multi-view image data acquisition method based on deep learning, comprising the following steps: S1. Collect image data from different perspectives simultaneously through an image acquisition device to generate multi-perspective original images; S2, preprocessing the collected multi-view original images; S3. Use the improved ResNet34 network to extract features from the preprocessed images, and optimize the feature extraction process through the attention mechanism to generate feature maps of images from different perspectives; S4, perform multi-level fusion of feature maps of images from different perspectives, introduce image spatial information, and generate comprehensive features of multi-perspective images; S5. Based on the comprehensive features of multi-view images, the improved generative adversarial network is used to reconstruct the image, output the final processed image data, and complete the image data collection.

[0019] In this implementation, the preprocessing includes denoising, geometric correction, color standardization and image alignment.

[0020] In this implementation, S3 specifically includes: S31, constructing an improved ResNet34 network, wherein the improved ResNet34 network includes: Input layer, input preprocessed image data; The convolution layer performs multi-scale feature extraction on the input image data at multiple scales to capture the local and global features of the image data. The convolution kernel sizes used in the convolution layer are 7×7 and 3×3. Batch normalization layer, which normalizes the output of each layer; The activation function layer performs nonlinear activation on the output of the convolutional layer through the ReLU activation function; The residual block directly adds the input to the output layer through a skip connection to keep the information flow smooth. The improved ResNet34 network introduces a residual connection with variable weights in the residual block based on the traditional ResNet34 network; Pooling layer, which applies max pooling after convolution, selects the maximum value in each 2×2 window and reduces the size of the feature map; The fully connected layer maps the features to the target output space by performing a weighted summation of the outputs of the previous layers and applying an activation function; Output layer, generating the final result of the improved ResNet34 network; S32, input the preprocessed image into the improved ResNet34 network for feature extraction, each residual block of the improved ResNet34 network includes two convolution layers, each convolution layer is followed by batch normalization and ReLU activation function, and the output is obtained through the convolution operation, and the output is fused with the input through the addition operation: ; ; in, represents the kth feature map extracted by improving the ResNet34 network, represents the weight of the kth convolution kernel, represents the kth bias term, represents the input image, (i, j) represents the pixel coordinates of each position in the image, m and n represent the size of the convolution kernel, represents the feature map extracted by the convolutional layer, represents the adaptive parameters; S33. Based on the attention mechanism, an attention weight vector is generated for each feature map through a fully connected layer, and the feature map is weighted to enhance the key information in the feature map and generate feature maps of pictures from different perspectives. The attention mechanism selects the degree of attention of the picture area in an adaptive manner: ; ; in, represents the attention weight vector of the kth feature map, σ represents the activation function, represents the weight of the fully connected layer, represents the bias of the fully connected layer, γ represents the adaptive weighting coefficient, Represents the weighted feature map.

[0021] In this implementation manner, the S4 specifically includes: S41. Fuse the feature maps of images from different perspectives: ; in, represents the fused features, Representation perspective The corresponding weight coefficient is obtained through the training process, where N represents the number of viewing angles; S42, by adding spatial information to improve the perception of the spatial structure of the image features, after the feature fusion, the spatial information and the fused features are connected to obtain the spatial features of the fused spatial information: ; in, represents the spatial features of fused spatial information, represents the feature connection operation, S(x,y) represents the spatial position information of each pixel in the image; S43. Combine multi-view image information, improve attention mechanism and spatial information, and finally generate comprehensive features of multi-view images: ; in, Represents the comprehensive features of multi-view images, Represents the spatial features of the fused spatial information.

[0022] In this implementation manner, S5 specifically includes: S51, constructing an improved generative adversarial network, wherein the improved generative adversarial network comprises: Convolutional layer, extracts local image features from the comprehensive features of multi-view images, captures low-level image information, and maps the input high-dimensional features to low-dimensional feature maps; Batch normalization layer, normalizes the output of the intermediate layer; Attention mechanism layer, which performs weighted processing on key areas in the image; The decoding layer maps the features optimized by the convolution layer and the attention mechanism layer back to the high-dimensional image space. The decoding layer restores the feature map to the final generated image through deconvolution operations, and generates the image through upsampling and transposed convolution. Generator output layer, outputs the generated image; The discriminator is responsible for determining whether the image generated by the generator is close to the real image, comparing the difference between the real image and the generated image, and outputting a true or false score; S52, receiving the comprehensive features of the multi-view image, and inputting the comprehensive features of the multi-view image into the convolutional layer of the improved generative adversarial network: ; in, Indicates The output feature map of the layer, Indicates The weight matrix of the convolution kernel of the layer, Indicates The bias vector of the layer convolution, Indicates The output feature map of the layer, when When Represents the comprehensive features of the input multi-view image , Conv2D represents a two-dimensional convolution operation, Represents the comprehensive features of the input multi-view images; S53, based on output feature map Generate query and key matrices: ; Where Q represents the query matrix, K represents the key matrix, Indicates The output feature map of the layer, and represents the weight matrix; S54, by calculating the dot product of the query matrix and the key matrix, the attention weight matrix is ​​obtained, normalized, and the attention weight matrix A is combined with the input feature map Combined, we get the weighted output feature map: ; ; Among them, A represents the attention weight matrix, represents the transpose of the key matrix, D represents the dimension of query and key, Softmax represents normalization, represents the weighted output feature map, V represents the value matrix, The weight matrix representing the value matrix; S55. The feature map obtained from the attention mechanism is decoded into a picture output through a decoding layer, wherein the decoding layer uses upsampling and convolution operations and includes two stages: In the first stage, the deconvolution layer is used to upsample the input feature map and restore the spatial size of the image: ; in, represents the restored image, DeConv represents the deconvolution operation, and Represent the weight matrix and bias term of deconvolution respectively; In the second stage, the restored image is processed through the activation function and the decoded image is output: ; Among them, Y represents the processed image, and tanh represents the nonlinear activation function; S56. Input the decoded image into a discriminator, wherein the discriminator includes a convolutional layer, an activation function, and a fully connected layer. The output of the discriminator is a value between 0 and 1: ; Where D(X) represents the output of the discriminator; S57. Use cross entropy loss to construct the loss function of the discriminator: ; Among them, D(X) represents the output of the discriminator for the real picture, D(Y) represents the output of the discriminator for the generated picture, X represents the real picture, and Y represents the generated picture. represents the data distribution, represents the generated distribution, represents the expectation of the data distribution, represents the expectation of the generated distribution, represents the loss function of the discriminator; S58. Use back propagation and gradient descent to update the parameters of the generator and discriminator, minimize the difference between the real image and the generated image, and finally output the generated image data.

[0023] Embodiment 1: In order to verify the feasibility of the present invention in implementation, the present invention is applied to the image data acquisition system of an industrial automation production line. The production line is responsible for the manufacture and assembly of a series of complex mechanical parts, including key parts such as gears and bearings. These parts have complex shapes, smooth surfaces and high processing accuracy requirements. Traditional image acquisition methods rely on single-view camera equipment, which cannot fully capture the detailed features of parts, especially in the case of complex surfaces or multi-dimensional perspectives. Insufficient image data can easily lead to missed or misjudgment of defect detection.

[0024] In order to overcome this challenge, the present invention adopts a technology that combines multi-view image acquisition with deep learning methods, and uses multiple camera devices to collect image data from different angles at the same time. After preprocessing, these image data are extracted using an improved ResNet34 network, and the feature map is optimized in combination with the attention mechanism, thereby achieving effective fusion of image features from different perspectives and extraction of depth information. By improving the generative adversarial network, the present invention not only improves the performance of image details, but also can complete high-precision three-dimensional reconstruction, thereby better supporting quality inspection tasks.

[0025] In this production line, an automated quality inspection system based on multi-view image data acquisition and processing is deployed. The system contains multiple high-definition cameras, which are installed at different positions of the production line to cover multiple perspectives of parts. The camera equipment is started synchronously, and the images are transmitted to the computer in real time for processing through high-speed data transmission. After preprocessing, the image data enters the improved ResNet34 network for feature extraction. In this process, the attention mechanism is used to optimize the key information in each perspective image to ensure that the system can focus on the key parts of the parts. Then, after all the collected image features are fused, spatial information is introduced, and image detail enhancement and three-dimensional reconstruction are performed through the improved generative adversarial network, and finally high-quality image data is output, providing accurate data support for subsequent quality inspection.

[0026] In order to verify the effectiveness of the method of the present invention, the system was tested several times in practical applications. The test object is a certain type of mechanical parts, the size of the parts is 20cm in diameter, 5cm in thickness, the surface coating is smooth, and it is easily affected by ambient light. By comparing with the traditional single-view image acquisition method, the following experimental results are obtained, as shown in Table 1.

[0027] Table 1 Experimental data comparison table ; According to the experimental results listed in the above table, it can be seen that the method of the present invention has significant advantages over the traditional image acquisition method in many aspects, especially in detection time, accuracy, missed detection rate, error judgment rate and three-dimensional reconstruction speed.

[0028] First, from the perspective of detection time, the method of the present invention shortens the time of the overall detection process. Taking GEAR-300 as an example, the detection time using the traditional method is 150 seconds, while the method of the present invention takes 80 seconds, which reduces the time by about 47%. In all experiments, the detection time using the method of the present invention is generally lower than that of the traditional method, indicating the advantage of the present invention in processing efficiency. This advantage is mainly attributed to the parallel acquisition of multi-view images and the high efficiency of feature extraction by the deep learning model, which improves the response speed and real-time performance of the system.

[0029] Secondly, the improvement in accuracy is another highlight of the present invention. In the GEAR-300 experiment, the accuracy of the traditional method was 85%, while the method of the present invention increased it to 98%. This improvement in accuracy stems from the comprehensive use of multi-view image data and the precise extraction of image features by the improved ResNet34 network and attention mechanism. In the detection of complex parts, traditional methods can often only rely on a single perspective for judgment, which can easily lead to errors in defect judgment. The method of the present invention can more comprehensively identify the detailed features of parts through multi-view information fusion, thereby significantly improving the detection accuracy.

[0030] The method of the present invention also performs well in terms of missed detection rate and false positive rate. For example, when testing SHAFT-200, the missed detection rate of the traditional method was 11%, while after adopting the method of the present invention, the missed detection rate dropped to 1%; the false positive rate of the traditional method was 9%, while the method of the present invention dropped to 3%. This improvement is due to the fact that the present invention uses a comprehensive analysis of multi-view images and combines the attention mechanism to enhance the perception of key parts during the feature extraction process, so that the system can better capture the defect characteristics of parts and components, reducing the occurrence of missed detection and false positives.

[0031] In terms of 3D reconstruction time, when using traditional methods, it is impossible to provide detailed 3D reconstruction time data, but the method of the present invention can complete 3D reconstruction in just 20 seconds. For example, in the test of BEARING-500, the 3D reconstruction time was greatly shortened, indicating that the present invention not only has advantages in detection, but also has significant improvements in the real-time performance of image reconstruction. Traditional image processing methods often take a long time to complete 3D reconstruction, which will affect the overall production efficiency in an industrial production environment. The method of the present invention significantly improves the speed of 3D reconstruction through an improved generative adversarial network and optimized processing of image data.

[0032] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-view image data acquisition method based on deep learning, characterized in that: The steps include: S1. Collect image data from different perspectives simultaneously through an image acquisition device to generate multi-perspective original images; S2, preprocessing the collected multi-view original images; S3. Use the improved ResNet34 network to extract features from the preprocessed images, and optimize the feature extraction process through the attention mechanism to generate feature maps of images from different perspectives; S4, perform multi-level fusion of feature maps of images from different perspectives, introduce image spatial information, and generate comprehensive features of multi-perspective images; S5. Based on the comprehensive features of multi-view images, the improved generative adversarial network is used to reconstruct the image, output the final processed image data, and complete the image data collection.

2. According to the multi-view image data acquisition method based on deep learning in claim 1, it is characterized in that: The preprocessing includes denoising, geometric correction, color normalization and image alignment.

3. The multi-view image data acquisition method based on deep learning according to claim 1, characterized in that: The S3 specifically includes: S31, constructing an improved ResNet34 network, wherein the improved ResNet34 network includes: Input layer, input preprocessed image data; The convolution layer performs multi-scale feature extraction on the input image data at multiple scales to capture the local and global features of the image data. The convolution kernel sizes used in the convolution layer are 7×7 and 3×3. Batch normalization layer, which normalizes the output of each layer; The activation function layer performs nonlinear activation on the output of the convolutional layer through the ReLU activation function; The residual block directly adds the input to the output layer through a skip connection to keep the information flow smooth. The improved ResNet34 network introduces a residual connection with variable weights in the residual block based on the traditional ResNet34 network; Pooling layer, which applies max pooling after convolution, selects the maximum value in each 2×2 window and reduces the size of the feature map; The fully connected layer maps the features to the target output space by performing a weighted summation of the outputs of the previous layers and applying an activation function; Output layer, generating the final result of the improved ResNet34 network; S32, input the preprocessed image into the improved ResNet34 network for feature extraction, each residual block of the improved ResNet34 network includes two convolution layers, each convolution layer is followed by batch normalization and ReLU activation function, and the output is obtained through the convolution operation, and the output is fused with the input through the addition operation: ； ； in, represents the kth feature map extracted by improving the ResNet34 network, represents the weight of the kth convolution kernel, represents the kth bias term, represents the input image, (i, j) represents the pixel coordinates of each position in the image, m and n represent the size of the convolution kernel, represents the feature map extracted by the convolutional layer, represents the adaptive parameters; S33. Based on the attention mechanism, an attention weight vector is generated for each feature map through a fully connected layer, and the feature map is weighted to enhance the key information in the feature map and generate feature maps of pictures from different perspectives. The attention mechanism selects the degree of attention of the picture area in an adaptive manner: ； ； in, represents the attention weight vector of the kth feature map, σ represents the activation function, represents the weight of the fully connected layer, represents the bias of the fully connected layer, γ represents the adaptive weighting coefficient, Represents the weighted feature map.

4. The multi-view image data acquisition method based on deep learning according to claim 1, characterized in that: The S4 specifically includes: S41. Fuse the feature maps of images from different perspectives: ； in, represents the fused features, Representation perspective The corresponding weight coefficient is obtained through the training process, where N represents the number of viewing angles; S42, by adding spatial information to improve the perception of the spatial structure of the image features, after the feature fusion, the spatial information and the fused features are connected to obtain the spatial features of the fused spatial information: ; in, represents the spatial features of fused spatial information, represents the feature connection operation, S(x,y) represents the spatial position information of each pixel in the image; S43. Combine multi-view image information, improve attention mechanism and spatial information, and finally generate comprehensive features of multi-view images: ; in, Represents the comprehensive features of multi-view images, Represents the spatial features of the fused spatial information.

5. The multi-view image data acquisition method based on deep learning according to claim 1, characterized in that: The S5 specifically includes: S51, constructing an improved generative adversarial network, wherein the improved generative adversarial network comprises: Convolutional layer, extracts local image features from the comprehensive features of multi-view images, captures low-level image information, and maps the input high-dimensional features to low-dimensional feature maps; Batch normalization layer, normalizes the output of the intermediate layer; Attention mechanism layer, which performs weighted processing on key areas in the image; The decoding layer maps the features optimized by the convolution layer and the attention mechanism layer back to the high-dimensional image space. The decoding layer restores the feature map to the final generated image through deconvolution operations, and generates the image through upsampling and transposed convolution. Generator output layer, outputs the generated image; The discriminator is responsible for determining whether the image generated by the generator is close to the real image, comparing the difference between the real image and the generated image, and outputting a true or false score; S52, receiving the comprehensive features of the multi-view image, and inputting the comprehensive features of the multi-view image into the convolutional layer of the improved generative adversarial network: ; in, Indicates The output feature map of the layer, Indicates The weight matrix of the convolution kernel of the layer, Indicates The bias vector of the layer convolution, Indicates The output feature map of the layer, when When Represents the comprehensive features of the input multi-view image , Conv2D represents a two-dimensional convolution operation, Represents the comprehensive features of the input multi-view images; S53, based on output feature map Generate query and key matrices: ； Where Q represents the query matrix, K represents the key matrix, Indicates The output feature map of the layer, and represents the weight matrix; S54, by calculating the dot product of the query matrix and the key matrix, the attention weight matrix is ​​obtained, normalized, and the attention weight matrix A is combined with the input feature map Combined, we get the weighted output feature map: ； ； Among them, A represents the attention weight matrix, represents the transpose of the key matrix, D represents the dimension of query and key, Softmax represents normalization, represents the weighted output feature map, V represents the value matrix, The weight matrix representing the value matrix; S55. The feature map obtained from the attention mechanism is decoded into a picture output through a decoding layer, wherein the decoding layer uses upsampling and convolution operations and includes two stages: In the first stage, the deconvolution layer is used to upsample the input feature map and restore the spatial size of the image: ； in, represents the restored image, DeConv represents the deconvolution operation, and Represent the weight matrix and bias term of deconvolution respectively; In the second stage, the restored image is processed through the activation function and the decoded image is output: ； Among them, Y represents the processed image, and tanh represents the nonlinear activation function; S56. Input the decoded image into a discriminator, wherein the discriminator includes a convolutional layer, an activation function, and a fully connected layer. The output of the discriminator is a value between 0 and 1: ; Where D(X) represents the output of the discriminator; S57. Use cross entropy loss to construct the loss function of the discriminator: ； Among them, D(X) represents the output of the discriminator for the real picture, D(Y) represents the output of the discriminator for the generated picture, X represents the real picture, and Y represents the generated picture. represents the data distribution, represents the generated distribution, represents the expectation of the data distribution, represents the expectation of the generated distribution, represents the loss function of the discriminator; S58. Use back propagation and gradient descent to update the parameters of the generator and discriminator, minimize the difference between the real image and the generated image, and finally output the generated image data.

Images