Image Reconstruction Method, Image Detection Method, Device, Equipment, Vehicle, and Medium

Abstract

Embodiments of the present application provide an image reconstruction method, an image detection method, a device, a device, a vehicle, and a medium. Multiple first encoding vectors of an original image are obtained from the original image. Since the dimension corresponding to each first encoding vector is lower than the dimension of a second encoding vector, and the second encoding vector includes the global features of the original image, the computational complexity in the quantization stage can be reduced. Also, since each first encoding vector has at least partially different features, after obtaining quantization vectors according to the first encoding vectors and merging the quantization vectors corresponding to each first encoding vector, the global features of the original image can be included, so that image reconstruction can be achieved based on the merged vectors, improving the image reconstruction efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to an image reconstruction method, an image detection method, a device, a device, a vehicle, and a medium. Background Art

[0002] In the process of industrial production and manufacturing, image anomaly detection is an important means to achieve product quality control. The reconstruction method is widely used in image anomaly detection. The reconstruction method refers to reconstructing the original image to obtain a reconstructed image, and determining whether the original image is abnormal based on the difference between the reconstructed image and the original image.

[0003] Currently, the original image can be reconstructed by a Vector Quantized Variational Auto-Encoder (VQVAE).

[0004] However, in the process of reconstructing the original image by the variational quantization auto-encoder, the amount of calculation is large, resulting in low reconstruction efficiency. Summary of the Invention

[0005] Embodiments of this application provide an image reconstruction method, an image detection method, a device, a device, a vehicle, and a medium to improve the reconstruction efficiency.

[0006] In a first aspect, an embodiment of this application provides an image reconstruction method, including:

[0007] Obtaining a plurality of first coding vectors of the original image from the original image, where there are at least partially different features between each of the first coding vectors, and the dimension corresponding to each of the first coding vectors is lower than the dimension of a second coding vector, and the second coding vector includes the global feature of the original image;

[0008] Obtaining quantization vectors according to the first coding vectors;

[0009] Merging the quantization vectors corresponding to each of the first coding vectors to obtain a merged vector;

[0010] Reconstructing the original image according to the merged vector to obtain a reconstructed image.

[0011] In some optional embodiments, the obtaining a plurality of first coding vectors of the original image from the original image includes:

[0012] Respectively extracting features from the original image through a plurality of encoders in the reconstruction model, and encoding the extracted features to obtain a plurality of first coding vectors of the original image.

[0013] In some alternative embodiments, obtaining a quantization vector according to the first encoding vector includes:

[0014] Calculating the similarity between the first encoding vector and the codebook vector through a quantizer in the reconstruction model, and determining the quantization vector corresponding to the first encoding vector according to the similarity.

[0015] In some alternative embodiments, reconstructing the original image according to the merging vector to obtain a reconstructed image includes:

[0016] Reconstructing the original image according to the merging vector through a decoder in the reconstruction model to obtain a reconstructed image.

[0017] In some alternative embodiments, the decoder is established based on a first reconstruction loss function and a second reconstruction loss function. The first reconstruction loss function is the loss function corresponding to the abnormal annotation area of the training image, and the second reconstruction loss function is the loss function corresponding to the non-annotated area of the training image. The first reconstruction loss function is greater than the second reconstruction loss function.

[0018] In some alternative embodiments, obtaining multiple first encoding vectors of the original image according to the original image includes:

[0019] Obtaining the feature vector of the original image;

[0020] Encoding the feature vector to obtain multiple first encoding vectors of the original image.

[0021] In some alternative embodiments, encoding the feature vector to obtain multiple first encoding vectors of the original image includes:

[0022] Encoding the feature vector through multiple encoders in the reconstruction model respectively to obtain multiple first encoding vectors of the original image.

[0023] In a second aspect, the present application provides an image detection method, including:

[0024] Obtaining the reconstructed image reconstructed by the image reconstruction method described in the first aspect;

[0025] Determining whether the original image is abnormal according to the reconstructed image.

[0026] In some alternative embodiments, determining whether the original image is abnormal according to the reconstructed image includes:

[0027] Comparing the reconstructed image and the original image to obtain a comparison result;

[0028] Determine whether the original image is abnormal according to the comparison result.

[0029] In some optional embodiments, comparing the reconstructed image and the original image to obtain a comparison result includes:

[0030] Using an image segmentation model to compare the reconstructed image and the original image to obtain a comparison result.

[0031] In a third aspect, the present application provides an image reconstruction device, including:

[0032] An encoding module, configured to obtain a plurality of first encoding vectors of the original image according to the original image. There are at least partially different features between each of the first encoding vectors, and the dimension corresponding to each of the first encoding vectors is lower than the dimension of a second encoding vector, where the second encoding vector includes the global features of the original image;

[0033] A quantization module, configured to obtain quantization vectors according to the first encoding vectors;

[0034] A merging module, configured to merge the quantization vectors corresponding to each of the first encoding vectors to obtain a merged vector;

[0035] A reconstruction module, configured to reconstruct the original image according to the merged vector to obtain a reconstructed image.

[0036] In a fourth aspect, the present application provides an image detection device, including:

[0037] An acquisition module, configured to acquire the reconstructed image reconstructed by the image reconstruction method described in the first aspect;

[0038] A determination module, configured to determine whether the original image is abnormal according to the reconstructed image.

[0039] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0040] The memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above method.

[0042] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above method.

[0043] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above method.

[0044] The image reconstruction method, image detection method, device, equipment, vehicle and medium provided by the embodiments of the present application obtain multiple first coding vectors of the original image from the original image. Since the dimension corresponding to each first coding vector is lower than that of the second coding vector, and the second coding vector includes the global features of the original image, the computational complexity in the quantization stage can be reduced. Moreover, since each first coding vector has at least partially different features, after obtaining the quantization vectors according to the first coding vectors and merging the quantization vectors corresponding to each first coding vector, the global features of the original image can be included, so that image reconstruction can be realized based on the merged vectors, and the image reconstruction efficiency can be improved. Brief Description of the Drawings

[0045] The drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0046] Figure 1 It is a schematic flowchart of the image reconstruction method provided by the present application;

[0047] Figure 2 It is a schematic diagram of image reconstruction provided by the present application;

[0048] Figure 3 It is a schematic flowchart of the image detection method provided by the present application;

[0049] Figure 4 It is a schematic diagram of image detection provided by the present application;

[0050] Figure 5 It is an architecture diagram of the image segmentation model provided by the present application;

[0051] Figure 6 It is a schematic structural diagram of the image reconstruction device provided by the present application;

[0052] Figure 7 It is a schematic structural diagram of the image detection device provided by the present application;

[0053] Figure 8 It is a schematic structural diagram of the electronic device provided by the present application.

[0054] Through the above-mentioned drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0055] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] In industrial production and manufacturing processes, image anomaly detection is an important means to achieve product quality control. The reconstruction method is widely used for image anomaly detection. The reconstruction method reconstructs the original image through a pre-trained model to obtain a reconstructed image. Since the pre-trained model is trained based on normal samples, it can capture the features of normal samples. If an abnormal sample is reconstructed, it cannot well capture the abnormal features, resulting in an increase in the difference between the reconstructed image and the original image. Therefore, when the difference between the reconstructed image and the original image is large, the original image is regarded as abnormal.

[0057] Exemplarily, the original image can be reconstructed by a Variational Auto-Encoder (VAE). The Variational Auto-Encoder is a reconstruction (also known as generative) model that can extract the latent features of the input data and represent these latent features in the form of an encoded vector (also known as a latent vector). Then, the encoded vector can be used to reconstruct the original data or generate new data samples. The Variational Auto-Encoder includes an Encoder and a Decoder. The Encoder maps the input data to a low-dimensional latent space. In the process of mapping the high-dimensional input data to the low-dimensional space, the Encoder attempts to retain the key information in the input data while removing redundancy and noise. Then, the Decoder reconstructs data similar to the input data based on the latent vector in the latent space.

[0058] However, when the Variational Auto-Encoder uses a relatively low-dimensional latent space to represent image data, it is easy to cause the loss of details of complex images and it is difficult to accurately reconstruct complex images.

[0059] Exemplarily, the original image can also be reconstructed by a Vector Quantized Variational AutoEncoder (VQVAE). The variational quantized autoencoder is a reconstruction model that combines variational autoencoder and vector quantization techniques. The variational quantized autoencoder not only includes an encoder and a decoder, but also a quantizer. The encoder maps the original image to a latent space and outputs an encoded vector. The quantizer can map the encoded vector to the nearest neighbor codebook vector in the codebook to obtain a discrete quantized vector. The decoder can better capture and represent discrete features in the image, such as edges, textures, etc., based on the discrete quantized vector, so as to be able to reconstruct an image similar to the original image.

[0060] However, the variational quantized autoencoder needs to map the encoded vector to the nearest neighbor codebook vector in the codebook, resulting in a large amount of computation and low reconstruction efficiency.

[0061] Therefore, this application proposes an image reconstruction method. Multiple first encoded vectors of the original image are obtained according to the original image. Since the dimension corresponding to each first encoded vector is lower than that of the second encoded vector, and the second encoded vector includes the global features of the original image, the amount of computation in the quantization stage can be reduced. And because each first encoded vector has at least partially different features, after merging the quantized vectors corresponding to each first encoded vector, the global features of the original image can be included, so as to be able to realize the reconstruction of the image.

[0062] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0063] Figure 1 It is a schematic flowchart of the image reconstruction method provided by this application, as Figure 1 shown, the method includes:

[0064] S101. Obtain multiple first encoded vectors of the original image according to the original image. There are at least partially different features between each first encoded vector, and the dimension corresponding to each first encoded vector is lower than that of the second encoded vector, and the second encoded vector includes the global features of the original image.

[0065] Exemplarily, an image usually exists in the form of pixels. Since an image usually consists of a large number of pixels, the image can be regarded as a high-dimensional data point. The encoded vector is obtained by converting the original image into a low-dimensional representation, which may involve compression and denoising to extract various feature information such as the texture, color, and shape of the original image.

[0066] Exemplarily, the original image can be mapped to a low-dimensional continuous latent space, which includes multiple points, and each point corresponds to an encoding vector, representing a compressed representation of the features. Therefore, by performing feature extraction and encoding processes on the original image, the original image can be converted into a low-dimensional encoding vector.

[0067] In some embodiments, the method of the present application can be performed by a reconstruction model.

[0068] In some alternative embodiments, the reconstruction model may include an encoder, through which the original image is subjected to feature extraction, and the extracted features are encoded to obtain a second encoding vector. Since the original image is subjected to feature extraction and encoding by one encoder, the second encoding vector can include the global features of the original image. The global features here can be understood as including all key information of the original image, and the key information of the original image can include important attributes such as the main content, structure, texture, and color of the image.

[0069] In some alternative embodiments, the reconstruction module may include multiple encoders. As Figure 2 shown, the original image can be subjected to feature extraction by multiple encoders respectively, and after encoding the extracted features, multiple first encoding vectors of the original image are obtained. After each encoder performs feature extraction and encoding on the original image, a first encoding vector of the original image can be obtained, and multiple first encoding vectors of the original image can be obtained through multiple encoders. Since the original image is processed by multiple encoders respectively, at least part of the feature information of the original image corresponding to each encoder can be different. Therefore, there are at least partially different features between each first encoding vector, so as to reduce the dimension of the first encoding vector output by each encoder. Correspondingly, the dimension of the first encoding vector can be lower than the dimension of the second encoding vector.

[0070] For example, when the number of encoders is two, the dimension of the first encoding vector output by each encoder can be half of the dimension of the second encoding vector.

[0071] Exemplarily, the number of encoders can be determined according to the actual situation, such as two, three, etc.

[0072] Exemplarily, the first encoding vector may include multiple vectors of the same dimension, and each vector may focus on different aspects or features of the data. For example, one vector may capture the color information of the image, while another vector may capture the shape information of the image.

[0073] Exemplarily, the reconstruction model can be, for example, a variational quantization autoencoder.

[0074] In some embodiments, considering the limited receptive field of the reconstruction model, it is difficult to capture the long-term dependencies in the long-sequence data when processing long-sequence data. Therefore, the feature vector of the original image can be obtained first, and the feature vector can be encoded to obtain multiple first encoding vectors of the original image, so as to solve the problem that the reconstruction model may not be able to process long-sequence data.

[0075] In some alternative embodiments, the feature vector of the original image can be obtained by a deep learning model for feature extraction of the original image. The deep learning module can include, for example, the Transformer architecture, which can capture the long-range dependencies and local features in the image through techniques such as the Self-Attention Mechanism and Positional Encoding.

[0076] In some alternative embodiments, after obtaining the feature vector of the original image, multiple encoders in the reconstruction model can be used to encode the feature vector respectively to obtain multiple first encoding vectors of the original image, so as to reduce the dimension of the first encoding vector output by each encoder.

[0077] S102. Obtain the quantization vector according to the first encoding vector.

[0078] It should be noted that the quantization vector can better capture and represent the discrete features in the image, such as edges, textures, etc. Therefore, in this step, the quantization vector is obtained according to the first encoding vector.

[0079] In some embodiments, the reconstruction model includes a quantizer, which calculates the similarity between the first encoding vector and the codebook vector through the quantizer in the reconstruction model, and determines the quantization vector corresponding to the first encoding vector according to the similarity, so as to obtain the quantization vector corresponding to the distance of the first encoding vector.

[0080] Specifically, calculate the similarity between each first encoding vector and the codebook vector to determine the quantization vector corresponding to each first encoding vector.

[0081] Exemplarily, taking a first encoding vector as an example, the similarity between the first encoding vector and each codebook vector in the codebook can be calculated, and the codebook vector with the maximum similarity to the first encoding vector is used as the quantization vector corresponding to the first encoding vector, as Figure 2 shown.

[0082] The codebook is generated during the training process of the reconstruction model. During the training process of the reconstruction model, the reconstruction model needs to learn how to encode the input data (such as an image) into a discrete representation in the latent space and reconstruct the input data through these discrete representations. The codebook is a set containing discrete vectors used to represent features in the latent space.

[0083] S103. Combine the quantization vectors corresponding to each first encoding vector to obtain a combined vector.

[0084] Since there are at least partially different features among each first encoding vector, in this step, the quantization vectors corresponding to each first encoding vector are combined to obtain a combined vector, as Figure 2 shown, so that the combined vector can contain the global characteristics of the original image, facilitating more accurate reconstruction of the original image subsequently.

[0085] In some alternative embodiments, multiple first encoding vectors can be concatenated or stacked along a specific dimension.

[0086] For example, multiple first encoders can be concatenated or stacked in the channel direction. In deep learning, in models such as variational quantization autoencoders, image data is usually decomposed into multiple channels, and each channel represents different features or color components of the image. For example, the three color channels of an RGB (Red Green Blue) image. The encoding vectors are high-dimensional representations extracted through the encoder network based on these channel features. Combining the encoding vectors in the channel direction means concatenating these high-dimensional representations along the channel dimension to form a new vector containing more feature information, which helps the reconstruction model better capture and utilize the multi-scale and multi-level features in the image, thereby improving the quality of image generation or encoding.

[0087] For example, two encoding vectors z1 and z2, which have shapes (batch_size, height, width, channels1) and (batch_size, height, width, channels2) respectively. The concatenate operation can be used to combine them along the channel dimension to form a new encoding vector, whose shape can be (batch_size, height, width, channels1 + channels2).

[0088] S104. Reconstruct the original image according to the combined vector to obtain a reconstructed image.

[0089] In some embodiments, the reconstruction model includes a decoder, and the original image is reconstructed according to the combined vector through the decoder in the reconstruction model to obtain a reconstructed image, as Figure 2as shown

[0090] Exemplarily, the decoder can convert a low-dimensional discrete representation (quantized vector) into a high-dimensional original data space (such as an image). For example, the decoder receives the quantized vector and processes it through a series of neural network layers, which gradually expand the low-dimensional latent representation into a high-dimensional output. In an image reconstruction task, the last few layers of the decoder usually use deconvolution operations to generate an output of the same size as the input image.

[0091] In some alternative embodiments, the decoder is established based on a first reconstruction loss function and a second reconstruction loss function. The first reconstruction loss function is the loss function corresponding to the abnormal annotation region of the training image, and the second reconstruction loss function is the loss function corresponding to the unannotated region of the training image. The first reconstruction loss function is greater than the second reconstruction loss function. By reducing the reconstruction loss of the abnormal annotation region and increasing the reconstruction loss of the unannotated region, the original image can be reconstructed more accurately and the reconstruction effect can be improved.

[0092] Exemplarily, the training image can be first input into a pre-established reconstruction model, and the reconstructed image output by the reconstruction model is displayed on an interactive user interface (User Interface, UI). Subsequently, manual annotation is performed on the reconstructed image displayed on the user interface to indicate which regions are normal (i.e., unannotated regions) and which regions are abnormal (i.e., abnormal annotation regions). Subsequently, the reconstruction model can be adjusted based on the manually annotated data. For the regions annotated as normal (unannotated regions), the model will try to reduce the reconstruction loss to better match these normal features. For the regions annotated as abnormal (abnormal annotation regions), the reconstruction model may try to increase the reconstruction loss so as to more accurately identify these abnormal features in the future. By training in this way combining supervised and unsupervised methods, the reconstruction model can gradually learn the information considered abnormal by humans, thereby improving the accuracy and robustness of the reconstruction model for anomaly detection.

[0093] The image reconstruction method provided by the embodiments of the present application obtains a plurality of first encoding vectors of the original image from the original image. Since the dimension corresponding to each first encoding vector is lower than that of the second encoding vector, and the second encoding vector includes the global features of the original image, the computational complexity in the quantization stage can be reduced. And since each first encoding vector has at least partially different features, after combining the quantization vectors corresponding to each first encoding vector, the global features of the original image can be included, thereby enabling the reconstruction of the image.

[0094] Figure 3 is a schematic flow chart of the image detection method provided by the present application, as Figure 3 shown, the method includes:

[0095] S201. Obtain the reconstructed image reconstructed by the above reconstruction method.

[0096] S202. Determine whether the original image is abnormal according to the reconstructed image.

[0097] In some alternative embodiments, after obtaining the reconstructed image, compare the reconstructed image with the original image to obtain a comparison result, and determine whether the original image is abnormal according to the comparison result, so as to realize the abnormality detection of the original image.

[0098] In some alternative embodiments, an image segmentation model can be used to compare the reconstructed image with the original image to obtain a comparison result, so as to more clearly display the difference region between the original image and the reconstructed image.

[0099] Exemplarily, the image segmentation model can be the UnetPP architecture. This network architecture combines large-scale and small-scale information through skip connections and downsampling, so that small targets and large targets can be effectively distinguished, ensuring the accuracy of segmentation. The input is the original image and the reconstructed image, and the output is a single-channel picture, such as Figure 4 the right figure in, where black represents the different regions and white represents the regions without differences. Compared with the actual different regions, such as Figure 4 the left figure in, the different regions can be more clearly displayed.

[0100] The UnetPP architecture adopts an encoder-decoder structure and introduces multi-scale feature fusion and denser skip connections. As Figure 5 shown, through the collaborative action of components such as convolution, upsampling, downsampling, and skip connections, the effective extraction and fusion of image features are realized, thereby improving the accuracy of image segmentation and reducing errors.

[0101] Convolution refers to extracting local features (such as edges, textures, etc.) from the input data through the sliding calculation of the convolution kernel. Each convolution kernel only focuses on a local area of the input data. Upsampling is used in the decoder part. The upsampling operation is used to gradually restore the spatial resolution of the feature map to make it close to the size of the input image. Through upsampling, the network can retain more detailed information, which helps to more accurately identify different regions in the image. Downsampling is used in the encoder part. The downsampling operation extracts higher-level features by reducing the spatial dimension of the feature map. Downsampling helps to reduce the video memory and computational amount, and at the same time increases the receptive field, enabling the network to extract features in a larger image range, thereby improving the accuracy of segmentation. The skip connection means directly connecting the feature map in the encoder to the corresponding layer in the decoder, realizing multi-scale feature fusion. Through the skip connection, the network can fuse the feature maps in the downsampling process during the upsampling process, thereby restoring more detailed information and reducing the segmentation error.

[0102] In some alternative embodiments, there are also errors in the reconstruction of the normal image. To avoid the influence of the reconstruction error of the normal region on the judgment of the abnormal region, after obtaining the reconstructed image, the reconstructed image can be processed by a noise filter, which can identify and process the abnormalities or noises in the reconstructed image, so as to more accurately determine whether the original image is abnormal.

[0103] Exemplarily, a noise filter can be trained with labeled data, where the labels indicate which regions are normal and which are abnormal, enabling the filter to learn how to effectively identify and process abnormal regions.

[0104] In some alternative embodiments, some image comparison tools can be used to compare the reconstructed image and the original image.

[0105] The image detection method provided by the embodiments of the present application determines whether the original image is abnormal by means of the reconstructed image.

[0106] Figure 6 It is a schematic structural diagram of an image reconstruction device provided by the present application, as Figure 6 shown, the image reconstruction device 10 provided in this embodiment includes:

[0107] An encoding module 11, configured to obtain multiple first encoding vectors of the original image according to the original image. There are at least partially different features between each first encoding vector, and the dimension corresponding to each first encoding vector is lower than the dimension of the second encoding vector, where the second encoding vector includes the global feature of the original image;

[0108] A quantization module 12, configured to obtain quantization vectors according to the first encoding vectors;

[0109] A merging module 13, configured to merge the quantization vectors corresponding to each first encoding vector to obtain a merged vector;

[0110] A reconstruction module 14, which reconstructs the original image according to the merged vector to obtain a reconstructed image.

[0111] In some alternative embodiments, the encoding module 11 is specifically configured to respectively perform feature extraction on the original image through multiple encoders in the reconstruction model, and encode the extracted features to obtain multiple first encoding vectors of the original image.

[0112] In some alternative embodiments, the quantization module 12 is specifically configured to calculate the similarity between the first encoding vector and the codebook vector through a quantizer in the reconstruction model, and determine the quantization vector corresponding to the first encoding vector according to the similarity.

[0113] In some alternative embodiments, the reconstruction module 14 is specifically configured to reconstruct the original image according to the merged vector through the decoder in the reconstruction model to obtain a reconstructed image.

[0114] In some alternative embodiments, the decoder is established based on a first reconstruction loss function and a second reconstruction loss function. The first reconstruction loss function is the loss function corresponding to the abnormal annotation area of the training image, and the second reconstruction loss function is the loss function corresponding to the non-annotated area of the training image. The first reconstruction loss function is greater than the second reconstruction loss function.

[0115] In some alternative embodiments, the encoding module 11 is specifically configured to obtain the feature vector of the original image; encode the feature vector to obtain multiple first encoding vectors of the original image.

[0116] In some alternative embodiments, the encoding module 11 is specifically configured to encode the feature vector through multiple encoders in the reconstruction model respectively to obtain multiple first encoding vectors of the original image.

[0117] The image reconstruction device provided in this embodiment can execute the image reconstruction method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0118] Figure 7 It is a schematic structural diagram of the image detection device provided in this application. As Figure 7 shown, the image detection device 20 provided in this embodiment includes:

[0119] An acquisition module 21, configured to acquire the reconstructed image reconstructed by the above reconstruction method;

[0120] A determination module 22, configured to determine whether the original image is abnormal according to the reconstructed image.

[0121] In some alternative embodiments, the determination module 22 is specifically configured to compare the reconstructed image and the original image to obtain a comparison result; determine whether the original image is abnormal according to the comparison result.

[0122] In some alternative embodiments, the determination module 22 is specifically configured to use an image segmentation model to compare the reconstructed image and the original image to obtain a comparison result.

[0123] The image detection device provided in this embodiment can execute the image detection method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0124] Figure 8 It is a schematic structural diagram of the electronic device provided in this application. As Figure 8As shown in the figure, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus.

[0125] In the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned method.

[0126] For the specific implementation process of the processor 501, reference can be made to the above method embodiment. The implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0127] In the above embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated: DSP), application-specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0128] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0129] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0130] This application also provides a vehicle, including the above-mentioned electronic device.

[0131] This application also provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0132] The present application also provides a computer-readable storage medium having computer-executable instructions stored therein, and when the processor executes the computer-executable instructions, the above-described method is implemented.

[0133] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present invention. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. An image reconstruction method, characterized in that, including: Feature extraction is respectively performed on the original image through multiple encoders, and the extracted features are encoded to obtain multiple first coding vectors of the original image. The feature information of the original image corresponding to each encoder is at least partially different, there are at least partially different features between each of the first coding vectors, the dimension corresponding to each of the first coding vectors is lower than the dimension of the second coding vector, and the second coding vector includes the global features of the original image; Determine the similarity between the first coding vector and the codebook vector, and determine the quantization vector corresponding to the first coding vector according to the similarity; Merge the quantization vectors corresponding to each of the first coding vectors to obtain a merged vector; Reconstruct the original image according to the merged vector to obtain a reconstructed image.

2. The method according to claim 1, wherein The reconstructing the original image according to the merged vector to obtain a reconstructed image includes: Reconstruct the original image according to the merged vector through a decoder in a reconstruction model to obtain a reconstructed image.

3. The method according to claim 2, wherein The decoder is established based on a first reconstruction loss function and a second reconstruction loss function. The first reconstruction loss function is the loss function corresponding to the abnormal annotation area of the training image, the second reconstruction loss function is the loss function corresponding to the non-annotated area of the training image, and the first reconstruction loss function is greater than the second reconstruction loss function.

4. An image detection method, characterized in that, including: Obtain the reconstructed image reconstructed by the image reconstruction method according to any one of claims 1-3; Determine whether the original image is abnormal according to the reconstructed image.

5. The method according to claim 4, characterized in that, The determining whether the original image is abnormal according to the reconstructed image includes: Compare the reconstructed image and the original image to obtain a comparison result; Determine whether the original image is abnormal according to the comparison result.

6. The method according to claim 5, wherein The comparing the reconstructed image and the original image to obtain a comparison result includes: Use an image segmentation model to compare the reconstructed image and the original image to obtain a comparison result.

7. An image reconstruction device, characterized in that, including: An encoding module, configured to respectively perform feature extraction on the original image through multiple encoders, and encode the extracted features to obtain multiple first coding vectors of the original image. The feature information of the original image corresponding to each encoder is at least partially different, there are at least partially different features between each of the first coding vectors, the dimension corresponding to each of the first coding vectors is lower than the dimension of the second coding vector, and the second coding vector includes the global features of the original image; A quantization module, configured to determine the similarity between the first coding vector and the codebook vector, and determine the quantization vector corresponding to the first coding vector according to the similarity; A merging module, configured to merge the quantization vectors corresponding to each of the first coding vectors to obtain a merged vector; A reconstruction module, configured to reconstruct the original image according to the merged vector to obtain a reconstructed image.

8. An image detection device, characterized in that, including: An obtaining module, configured to obtain the reconstructed image reconstructed by the image reconstruction method according to any one of claims 1-3; A determining module, configured to determine whether the original image is abnormal according to the reconstructed image.

9. An electronic device, characterized in that, including: A memory, a processor; The memory stores computer execution instructions; The processor executes the computer-executable instructions stored in the memory, such that the processor performs the method according to any one of claims 1-6.

10. A vehicle, characterized in that, An electronic device according to claim 9.

11. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1-6.

12. A computer program product, characterized in that, A computer program, which when executed by a processor, implements the method according to any one of claims 1-6.

Images