Image data generation method, anomaly detection method, electronic device, and storage medium

Abstract

The application provides an image data generation method, an anomaly detection method, an electronic device and a storage medium. The image data generation method comprises the following steps: obtaining a three-dimensional model of a target object, a real defect image and a normal image; generating a synthetic defect image and label information of the synthetic defect image based on the three-dimensional model and the real defect image, wherein the label information of the synthetic defect image comprises distribution information of defects of the three-dimensional model in the synthetic defect image; performing simulation processing on the normal image of the target object based on a preset generative adversarial network model to generate a plurality of simulated defect images; and constructing an image fusion data set based on the plurality of real defect images, the plurality of synthetic defect images, the plurality of simulated defect images and the label information of each synthetic defect image in the plurality of synthetic defect images. The application reduces the dependence on the number of real defect images and reduces the data acquisition cost.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to an image data generation method, an anomaly detection method, an electronic device, and a storage medium. Background Technology

[0002] In industrial production, anomaly detection is a crucial step in ensuring product quality and production efficiency, and is widely used in industries such as manufacturing, electronics, and automotive. Traditional industrial anomaly detection methods rely on manual inspection, which is inefficient and prone to errors. Currently, with the rapid development of machine learning and artificial intelligence, machine learning technology has begun to be widely applied in the automated maintenance and operation of the railway sector. Industrial anomaly detection based on machine learning technology is gradually replacing manual inspection and has become the mainstream choice for industrial anomaly detection.

[0003] Industrial anomaly detection based on machine learning technology faces challenges due to its inherent lack of interpretability. The accuracy and recall of machine learning models (such as deep learning models) are heavily reliant on the quantity and quality of training samples. Therefore, acquiring a large amount of labeled sample data is necessary to train a highly accurate anomaly detection model. However, in this field, anomaly samples are relatively scarce and complex in actual production. Data collection and labeling are costly, and the lack of diversity and realism in the sample data makes it difficult to cover real-world anomaly scenarios. This results in limited model generalization ability, leading to low anomaly detection accuracy for anomaly detection models and consuming significant manpower and time. Summary of the Invention

[0004] In view of the above, it is necessary to propose an image data generation method, an anomaly detection method, electronic device and storage medium to solve the technical problems of existing anomaly detection methods, such as the lack of diversity and authenticity of sample data, difficulty in covering actual anomaly scenarios, limited model generalization ability, and low anomaly detection accuracy of anomaly detection model-based detection methods.

[0005] In a first aspect, this application provides an image data generation method, the method comprising: acquiring a three-dimensional model of a target object, a real defect image, and a normal image, wherein the real defect image refers to a real-world image of a target object with defects, and the normal image refers to a real-world image of a target object without defects; generating a synthetic defect image and annotation information of the synthetic defect image based on the three-dimensional model and the real defect image, wherein the annotation information of the synthetic defect image includes the distribution information of defects in the three-dimensional model in the synthetic defect image; performing simulation processing on the normal image of the target object based on a preset generative adversarial network model to generate multiple simulated defect images; and constructing an image fusion dataset based on the multiple real defect images, the multiple synthetic defect images, the multiple simulated defect images, and the annotation information of each of the multiple synthetic defect images.

[0006] In the image data generation method of the above embodiments, firstly, a three-dimensional model of the target object, a real defect image, and a normal image are acquired; then, based on the three-dimensional model and the real defect image, a synthetic defect image and annotation information of the synthetic defect image are generated, wherein the annotation information of the synthetic defect image includes the distribution information of the defects of the three-dimensional model in the synthetic defect image; next, the normal image of the target object is simulated based on a preset generative adversarial network model to generate multiple simulated defect images; finally, an image fusion dataset is constructed based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image among the multiple synthetic defect images. In this application, real defect images provide basic defect samples. By combining 3D models with real defect images, synthetic defect images with different defect types and distributions can be generated. Simulated defect images are generated using a generative adversarial network (GAN) model, further expanding the diversity of defect samples, reducing dependence on the number of real defect images, and lowering data acquisition costs. Furthermore, by fusing real, synthetic, and simulated defect images, a comprehensive image fusion dataset is constructed, enabling the dataset to cover a wider range of defect types and scenarios, improving its comprehensiveness and richness, and facilitating the training of anomaly detection models capable of recognizing various complex defect distributions. The generation of synthetic and simulated defect images significantly increases the number of training samples, providing sufficient training data for subsequent anomaly detection models. This allows the anomaly detection models to better learn defect features and recognition patterns during training, enabling them to more accurately identify and classify defects in various complex scenarios in practical applications, thereby improving the generalization ability and robustness of the anomaly detection models. The annotation information for synthetic and simulated defect images can be automatically generated, reducing the need for manual annotation of real defect images, significantly reducing the cost and time of data annotation, and improving the efficiency of data preparation. Based on this, the image data generation method of this application can construct a rich, diverse and efficient image fusion dataset by fusing multi-source data and automatically generating annotation information. This reduces the dependence on the number of real defect images, lowers the data acquisition cost, provides strong support for the training and performance improvement of anomaly detection models, and has good application scalability.

[0007] In some embodiments of this application, generating a synthetic defect image and its annotation information based on the 3D model and the real defect image includes: generating a defect texture map based on the real defect image; mapping the defect texture map onto the surface of the 3D model to obtain a simulated 3D model; and rendering the simulated 3D model using rendering technology to generate the synthetic defect image and its annotation information.

[0008] In some embodiments of this application, mapping the defect texture map onto the surface of the three-dimensional model to obtain a simulated three-dimensional model includes: mapping the surface of the three-dimensional model onto a two-dimensional plane; seamlessly editing the defect texture map to generate a seamless defect texture map; and mapping the seamless defect texture map onto the two-dimensional plane to obtain the simulated three-dimensional model.

[0009] In some embodiments of this application, after mapping the seamless defect texture map onto the two-dimensional plane, the method further includes: adjusting the layout information of the seamless defect texture map on the surface of the three-dimensional model based on preset adjustment parameters.

[0010] In some embodiments of this application, the step of rendering the simulated 3D model based on rendering technology to generate the synthetic defect image and the annotation information of the synthetic defect image includes: obtaining the rendering parameters of the simulated 3D model; rendering the simulated 3D model based on the rendering engine and the rendering parameters to generate the synthetic defect image, an identifier for each defect in the simulated 3D model, and related information for each defect; and generating the annotation information of the synthetic defect image based on the identifier for each defect and the related information for each defect.

[0011] In some embodiments of this application, before generating a defect texture map based on the real defect image, the method further includes: mapping a preset scene texture map onto the surface of the three-dimensional model, wherein the scene texture map includes a variety of different texture attributes.

[0012] In some embodiments of this application, before constructing an image fusion dataset based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each of the multiple synthetic defect images, the method further includes: converting the style of the synthetic defect images into a style similar to that of the real defect images based on the adversarial generative network model to obtain the converted synthetic defect images.

[0013] Secondly, this application also provides an anomaly detection method, the method comprising: constructing an image fusion dataset based on the image data generation method of the above embodiments; training a preset anomaly detection model based on the image fusion dataset to obtain a trained anomaly detection model; and performing anomaly detection on the target object based on the trained anomaly detection model.

[0014] In the anomaly detection method of this application, the number of training samples is expanded through image data generation methods, providing the anomaly detection model with more data to learn from during training. This allows the model to better grasp the patterns and characteristics of defect detection. The diverse datasets enable the anomaly detection model to adapt to different defect types and scenarios during training, improving its generalization ability in various complex situations encountered in practical applications and reducing false positives and false negatives. The trained anomaly detection model can quickly detect anomalies in target objects without requiring manual inspection, improving detection efficiency and ensuring production quality and efficiency. The anomaly detection model trained on a high-quality dataset has high detection accuracy, accurately identifying defects in target objects, which helps improve product quality and reduce losses caused by defects. In summary, the anomaly detection method of this application, by constructing a rich image fusion dataset and training an efficient anomaly detection model, can achieve efficient and accurate anomaly detection of target objects, possessing good application flexibility and scalability, and can meet the detection tasks of different scenarios and needs.

[0015] In some embodiments of this application, training a preset anomaly detection model based on the image fusion dataset to obtain a trained anomaly detection model includes: pre-training the preset anomaly detection model based on the multiple simulated defect images to obtain a first detection model; fine-tuning the first detection model based on the multiple synthetic defect images and the annotation information of each synthetic defect image to obtain a second detection model; and fine-tuning the second detection model based on the real defect images to obtain the trained anomaly detection model.

[0016] Thirdly, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the image data generation method as described in the above embodiments or the anomaly detection method as described in the above embodiments.

[0017] Fourthly, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image data generation method described in the above embodiments or the anomaly detection method described in the above embodiments. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0019] Figure 2This is a schematic flowchart of an image data generation method provided in an embodiment of this application.

[0020] Figure 3 This is a detailed flowchart illustrating an image data generation method provided in an embodiment of this application.

[0021] Figure 4 This is a flowchart illustrating an anomaly detection method provided in an embodiment of this application.

[0022] Component Symbol Explanation Electronic devices 10 Memory 11 Processor 12 The following detailed description, in conjunction with the accompanying drawings, will further illustrate this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0024] In the embodiments of this application, it should be noted that, unless otherwise expressly specified and limited, the word "for example" is used to indicate an example, illustration, or description. Any embodiment or design scheme described as "for example" in the embodiments of this application should not be construed as being better or more advantageous than other embodiments or design schemes. Specifically, the use of the word "for example" is intended to present the relevant concepts in a specific manner.

[0025] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0026] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. Furthermore, in the description of this application, "a plurality of" means two or more, unless otherwise expressly and specifically defined.

[0027] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0028] Please see Figure 1 This is a schematic diagram of the hardware structure of an electronic device 10 provided in an embodiment of this application.

[0029] This application provides an image data generation or anomaly detection method, which can be applied to one or more electronic devices 10. The electronic device 10 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0030] Specifically, the electronic device 10 is used to: acquire a 3D model of the target object, a real defect image, and a normal image, wherein the real defect image refers to a real-life image of the target object with defects, and the normal image refers to a real-life image of the target object without defects; generate a synthetic defect image and its annotation information based on the 3D model and the real defect image, wherein the annotation information of the synthetic defect image includes the distribution information of the defects in the 3D model in the synthetic defect image; perform simulation processing on the normal image of the target object based on a preset generative adversarial network model to generate a simulated defect image; and construct an image fusion dataset based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image in the multiple synthetic defect images.

[0031] Specifically, the electronic device 10 is also used to: construct an image fusion dataset based on the image data generation method; train a preset anomaly detection model based on the image fusion dataset to obtain a trained anomaly detection model; and perform anomaly detection on the target object based on the trained anomaly detection model.

[0032] In some embodiments of this application, the electronic device 10 can be communicatively connected to devices such as desktop computers, laptops, handheld computers, and cloud servers.

[0033] In some embodiments of this application, the electronic device 10 can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0034] In some embodiments of this application, the electronic device 10 may further include network devices and / or client devices. These network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, and a cloud server based on cloud computing, consisting of a large number of hosts or network servers.

[0035] In some embodiments of this application, the network where the electronic device 10 is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0036] Please see Figure 2 This is a schematic diagram of the steps of an image data generation method provided in an embodiment of this application.

[0037] The image data generation method of this application embodiment is applied in electronic devices, for example... Figure 1 The electronic device 10 in the process. Specifically, the image data generation method includes the following steps. Depending on different needs, the order of some steps in the flowchart can be changed, and some steps can be omitted.

[0038] Step S10: Obtain the 3D model of the target object, the real defect image, and the normal image.

[0039] In some embodiments of this application, the target object may be a workpiece that requires anomaly detection.

[0040] In some embodiments of this application, a three-dimensional model of the target object is constructed based on three-dimensional design drawings of actual industrial scenarios.

[0041] In some embodiments of this application, a real defect image refers to a photograph of a target object with defects. A normal image refers to a photograph of a target object without defects. The electronic device 10 can acquire real defect images and normal images using an industrial camera.

[0042] Step S11: Based on the 3D model and the real defect image, generate a synthetic defect image and its annotation information.

[0043] In some embodiments of this application, generating a synthetic defect image and its annotation information based on a 3D model and a real defect image specifically includes: generating a defect texture map based on the real defect image; mapping the defect texture map onto the surface of the 3D model to obtain a simulated 3D model; and rendering the simulated 3D model using rendering technology to generate the synthetic defect image and its annotation information.

[0044] In some embodiments of this application, mapping a defect texture map onto the surface of a 3D model to obtain a simulated 3D model includes: mapping the surface of the 3D model onto a 2D plane; seamlessly editing the defect texture map to generate a seamless defect texture map; and mapping the seamless defect texture map onto the 2D plane to obtain a simulated 3D model. This allows for the simulation of various defects on the surface of a target object, such as scratches, dents, and stains, making the appearance of the simulated 3D model closer to the target object in the real world, and improving the realism, detail richness, and flexibility of texture application in the simulation effect.

[0045] In some embodiments of this application, after mapping the seamless defect texture map onto a two-dimensional plane, the image generation method further includes: adjusting the layout information of the seamless defect texture map on the surface of the three-dimensional model based on preset adjustment parameters. For example, adjusting the position, orientation, and size of the seamless defect texture map on the surface of the three-dimensional model to simulate different defect distributions, which can more realistically simulate the wear and defects that naturally form on the target object during use and improve the overall realism of the simulated three-dimensional model.

[0046] In some embodiments of this application, rendering a simulated 3D model based on rendering technology to generate a synthetic defect image and its annotation information includes: obtaining rendering parameters of the simulated 3D model; rendering the simulated 3D model based on the rendering engine and rendering parameters to generate a synthetic defect image, an identifier for each defect in the simulated 3D model, and related information for each defect, such as location information, thereby simulating lighting in the actual detection environment and increasing the richness of the image fusion dataset to generate a high-quality synthetic defect image; generating annotation information for the synthetic defect image based on the identifier and related information for each defect, and exporting the annotation information to file formats such as JSON and XML. The annotation information for the synthetic defect image includes the distribution information of defects in the 3D model within the synthetic defect image, exhibiting good compatibility and scalability. The high-quality synthetic defect image and accurate annotation information provide a reliable foundation for subsequent anomaly detection and analysis.

[0047] In some embodiments of this application, before generating a defect texture map based on a real defect image, the image generation method further includes: mapping a preset scene texture map onto the surface of a 3D model, wherein the scene texture map includes different texture attributes, such as diffuse reflection, normal, metallicity, roughness, ambient occlusion, and other channels, thereby improving the realism of scene rendering.

[0048] It should be noted that the details on how to generate synthetic defect images and their annotation information based on 3D models and real defect images will be discussed later. Figure 3 The steps shown are described in detail, and will not be repeated here to avoid repetition.

[0049] Step S12: Based on the preset adversarial generative network model, the normal image of the target object is simulated to generate multiple simulated defect images.

[0050] In some embodiments of this application, the training data for the Generative Adversarial Network (GAN) model can be the generated synthetic defect image and its annotation information from step S11. When a small number of real defect images exist in actual industrial production, these real defect images can also be used as training data, thereby improving the realism of the simulated defect images generated by the GAN.

[0051] In the above embodiments, generating simulated defect images through GAN can effectively expand the scale of the image fusion dataset, provide more training samples for the anomaly detection model, make up for the difference between synthetic defect images and real defect images, and the generated simulated defect images can cover various types of defects, such as scratches, dents, stains, etc., and can generate defect samples outside the sampling data space, thereby improving the diversity of the defect dataset and helping to improve the generalization ability of the anomaly detection model.

[0052] Step S13: Construct an image fusion dataset based on multiple real defect images, multiple synthetic defect images, and multiple simulated defect images.

[0053] In some embodiments of this application, constructing an image fusion dataset based on multiple real defect images, multiple synthetic defect images, and multiple simulated defect images includes: converting the style of the synthetic defect images into a style similar to that of the real defect images based on an adversarial generative network model to obtain converted synthetic defect images, and then constructing an image fusion dataset based on multiple real defect images, multiple converted synthetic defect images, and multiple simulated defect images.

[0054] In the above embodiments, although the synthesized defect images can simulate some defect situations, they differ in style from real defect images. By using an adversarial generative network model to transform the style of the synthesized defect images into a style similar to that of real defect images, this gap can be effectively narrowed. This allows the synthesized defect images to more closely resemble the characteristics of images in real-world scenes while retaining the original defect features, thereby increasing data diversity. This enables the anomaly detection model to better identify and detect defects in different scenes and styles, improving the generalization ability of the anomaly detection model and reducing overfitting.

[0055] In the image data generation method of this application embodiment, firstly, a three-dimensional model of the target object, a real defect image, and a normal image are acquired; then, based on the three-dimensional model and the real defect image, a synthetic defect image and annotation information of the synthetic defect image are generated, wherein the annotation information of the synthetic defect image includes the distribution information of the defects of the three-dimensional model in the synthetic defect image; next, based on a preset generative adversarial network model, the normal image of the target object is simulated to generate multiple simulated defect images; finally, an image fusion dataset is constructed based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image in the multiple synthetic defect images. In this application, real defect images provide basic defect samples. By combining 3D models with real defect images, synthetic defect images with different defect types and distributions can be generated. Simulated defect images are generated using a generative adversarial network (GAN) model, further expanding the diversity of defect samples, reducing dependence on the number of real defect images, and lowering data acquisition costs. Furthermore, by fusing real, synthetic, and simulated defect images, a comprehensive image fusion dataset is constructed, enabling the dataset to cover a wider range of defect types and scenarios, improving its comprehensiveness and richness, and facilitating the training of anomaly detection models capable of recognizing various complex defect distributions. The generation of synthetic and simulated defect images significantly increases the number of training samples, providing sufficient training data for subsequent anomaly detection models. This allows the anomaly detection models to better learn defect features and recognition patterns during training, enabling them to more accurately identify and classify defects in various complex scenarios in practical applications, thereby improving the generalization ability and robustness of the anomaly detection models. The annotation information for synthetic and simulated defect images can be automatically generated, reducing the need for manual annotation of real defect images, significantly reducing the cost and time of data annotation, and improving the efficiency of data preparation. Based on this, the image data generation method of this application can construct a rich, diverse and efficient image fusion dataset by fusing multi-source data and automatically generating annotation information. This reduces the dependence on the number of real defect images, lowers the data acquisition cost, provides strong support for the training and performance improvement of anomaly detection models, and has good application scalability.

[0056] Please see Figure 3 This is a detailed flowchart of an image data generation method provided in an embodiment of this application.

[0057] The image data generation method of this application embodiment is applied in electronic devices, for example... Figure 1The electronic device 10 in the embodiment. This embodiment is a specific description of step S11 in the foregoing embodiment, further illustrating how to generate a synthetic defect image and annotation information of the synthetic defect image based on a 3D model and a real defect image. Specifically, it includes the following steps: Step S111: Generate a defect texture map based on the real defect image.

[0058] In some embodiments of this application, before generating the defect texture map, the electronic device 10 can perform preprocessing operations on the real defect image to improve the extraction effect of defect features. These preprocessing operations may include, but are not limited to, using Gaussian filtering to remove noise from the image and using histogram equalization to enhance image contrast. Further, image processing algorithms are employed to extract features such as the contour and texture of the defect. Common image processing algorithms include, but are not limited to, edge detection algorithms (e.g., Canny edge detection) and thresholding algorithms (e.g., Otsu thresholding).

[0059] Specifically, generating defect texture maps based on real defect images includes: editing real defect images using image editing software (such as Adobe, Photoshop, Blender, etc.) (e.g., cropping, resizing, color correction, etc.) to convert them into defect texture maps, thereby ensuring the quality and consistency of the defect texture maps. The defect texture maps can include different defect types, such as scratches and dents, and the defect texture maps should be compatible with the texture format of the 3D model. Common formats include PNG, JPEG, etc.

[0060] In some embodiments of this application, before generating a defect texture map based on a real defect image, the image generation method further includes: mapping a preset scene texture map onto the surface of a 3D model. The scene texture map includes different texture attributes, such as channels for diffuse reflection, normals, metallicity, roughness, and ambient occlusion, thereby improving the realism of the scene rendering.

[0061] Step S112: Map the defect texture map onto the surface of the 3D model to obtain the simulated 3D model.

[0062] Specifically, mapping defect texture maps onto the surface of a 3D model to obtain a simulated 3D model includes: mapping the surface of the 3D model onto a 2D plane; seamlessly editing the defect texture map to generate a seamless defect texture map, where a Photoshop plugin can be used to convert the defect texture map into a seamless defect texture map, thus ensuring that the seamless defect texture map can be tiled seamlessly without obvious seams; and mapping the seamless defect texture map onto a 2D plane to obtain a simulated 3D model, which can then simulate various defects on the surface of the target object, such as scratches, dents, and stains, making the appearance of the simulated 3D model closer to the target object in the real world, improving the realism, detail richness, and flexibility of texture application in the simulation effect.

[0063] In some embodiments of this application, after mapping the seamless defect texture map onto a two-dimensional plane, the image generation method further includes: adjusting the layout information of the edited defect texture map on the surface of the three-dimensional model based on preset adjustment parameters (such as position, direction, size, etc.). For example, adjusting the position, direction, and size of the seamless defect texture map on the surface of the three-dimensional model can simulate different defect distributions, which can more realistically simulate the wear and defects that naturally form on the target object during use and improve the overall realism of the simulated three-dimensional model.

[0064] Step S113: Render the simulated 3D model based on rendering technology to generate a synthetic defect image and annotation information of the synthetic defect image.

[0065] Specifically, the process of rendering a simulated 3D model using rendering technology to generate synthetic defect images and their annotation information includes: obtaining rendering parameters for the simulated 3D model; rendering the simulated 3D model based on the rendering engine and rendering parameters to generate synthetic defect images, identifiers for each defect in the simulated 3D model, and related information for each defect, such as location information. Rendering parameters include lighting conditions (e.g., ambient light, point light sources, spotlights) and scene conditions (e.g., changing background, light color, light intensity, etc.), thereby simulating lighting in the actual detection environment and enriching the image fusion dataset to generate high-quality synthetic defect images; generating annotation information for the synthetic defect images based on the identifiers and related information for each defect, and exporting the annotation information to file formats such as JSON and XML, which has good compatibility and scalability. The high-quality synthetic defect images and accurate annotation information provide a reliable foundation for subsequent anomaly detection and analysis. The annotation information for the synthetic defect images includes the distribution information of defects in the 3D model within the synthetic defect images.

[0066] In the above embodiments, professional image editing software is used to edit real defect images and convert them into defect texture maps compatible with the texture format of the 3D model. This ensures the quality and consistency of the defect texture maps, covering multiple defect types and providing high-quality materials for simulating defects on the surface of the 3D model. Preset scene texture maps are mapped onto the surface of the 3D model, enriching scene texture attributes and enhancing the realism of the rendering. Seamless editing and mapping of the defect texture maps simulates real defects, and adjusting layout information to simulate natural wear and tear, comprehensively improving the realism of the simulated 3D model. Based on rendering technology, the simulated 3D model is rendered according to rendering parameters to generate high-quality synthetic defect images. Annotation information for the synthetic defect images is generated based on the identifier and related information of each defect, along with precise annotation information, and exported in a universal format, providing a reliable data foundation for subsequent anomaly detection and analysis.

[0067] Please see Figure 4 This is a schematic diagram of the steps of an anomaly detection method provided in an embodiment of this application.

[0068] The image data generation method of this application embodiment is applied in electronic devices, for example... Figure 1 Electronic device 10. Specifically, the anomaly detection method includes the following steps. Depending on different requirements, the order of some steps in this flowchart can be changed, and some steps can be omitted.

[0069] Step S20: Construct an image fusion dataset.

[0070] In some embodiments of this application, an image fusion dataset is constructed based on the image data generation method described in the above embodiments. The image fusion dataset includes multiple real defect images of the target object, multiple synthetic defect images, multiple simulated defect images, and annotation information for each synthetic defect image. It covers defect data from different sources and types, providing more comprehensive defect feature coverage. This allows the anomaly detection model to encounter various defect situations during training, thereby improving the generalization ability and adaptability of the anomaly detection model.

[0071] Step S21: Train the preset anomaly detection model based on the image fusion dataset to obtain the trained anomaly detection model.

[0072] In some embodiments of this application, the preset anomaly detection model may be a model based on a convolutional neural network.

[0073] Specifically, training a pre-defined anomaly detection model based on an image fusion dataset to obtain a trained anomaly detection model includes: pre-training the pre-defined anomaly detection model based on multiple simulated defect images to obtain a first detection model; fine-tuning the first detection model based on multiple synthetic defect images and the annotation information of each synthetic defect image to obtain a second detection model; and fine-tuning the second detection model based on real defect images to obtain a trained anomaly detection model.

[0074] In some embodiments of this application, the pre-training stage utilizes simulated defect images to enable the anomaly detection model to initially learn defect features. Simulated defect images are generally easier to obtain and more numerous, allowing for rapid pre-training and laying the foundation for subsequent fine-tuning. The fine-tuning stage further optimizes the anomaly detection model using synthetic and real defect images, respectively. While synthetic and real defect images are relatively fewer in number, they are of higher quality. By fine-tuning based on pre-training, these high-quality data can be utilized more efficiently, improving overall training efficiency and ultimately resulting in a trained anomaly detection model with superior overall performance. Simulated, synthetic, and real defect images cover different types of defect features and distributions. By training on these different datasets, the anomaly detection model can learn a wider range of defect features, thereby better identifying various types of defects in practical applications and improving the generalization ability of the anomaly detection model. Image data from different sources reflects diverse detection scenarios and conditions. Simulated defect images may mimic specific detection environments, synthetic defect images combine 3D models and real defect features, while real defect images reflect actual production conditions. By training on data from these diverse scenarios, the anomaly detection model can adapt to different detection scenarios and exhibit better robustness. In the fine-tuning training phase using synthetic defect images, the annotation information from these images allows the anomaly detection model to learn the distribution and characteristics of defects more accurately, providing a clear learning objective and improving its accuracy in identifying defect locations and features. The final fine-tuning training phase uses real defect images, reflecting actual defect conditions in production. By fine-tuning on real data, the anomaly detection model can calibrate the features learned from simulated and synthetic defect images, further improving its detection accuracy in practical applications.

[0075] In the above embodiments, by training the anomaly detection model in stages, the advantages of different datasets can be fully utilized, the performance of the anomaly detection model can be gradually optimized, and the generalization ability, adaptability and detection accuracy of the anomaly detection model can be improved, which has good application flexibility and scalability.

[0076] Step S22: Perform anomaly detection on the target object based on the trained anomaly detection model.

[0077] In some embodiments of this application, anomaly detection of the target object is performed based on the trained anomaly detection model, which can obtain the identifier and related information of the defects of the target object, such as the type of anomaly (e.g., scratches, dents, cracks, etc.) and the specific location, providing detailed information for subsequent repair or processing.

[0078] In the anomaly detection method of this application, the number of training samples is expanded through image data generation methods, providing the anomaly detection model with more data to learn from during training. This allows the model to better grasp the patterns and characteristics of defect detection. The diverse datasets enable the anomaly detection model to adapt to different defect types and scenarios during training, improving its generalization ability in various complex situations encountered in practical applications and reducing false positives and false negatives. The trained anomaly detection model can quickly detect anomalies in target objects without requiring manual inspection, improving detection efficiency and ensuring production quality and efficiency. The anomaly detection model trained on a high-quality dataset has high detection accuracy, accurately identifying defects in target objects, which helps improve product quality and reduce losses caused by defects. In summary, the anomaly detection method of this application, by constructing a rich image fusion dataset and training an efficient anomaly detection model, can achieve efficient and accurate anomaly detection of target objects, possessing good application flexibility and scalability, and can meet the detection tasks of different scenarios and needs.

[0079] Combination Figure 1 As shown, in some embodiments of this application, the electronic device 10 includes, but is not limited to, a memory 11, a processor 12, and a computer program stored in the memory 11 and executable on the processor 12, such as an image data generation or anomaly detection program. When the computer program is executed by the processor, it implements the image data generation or anomaly detection method as described in the above embodiments.

[0080] Figure 1 Only the electronic device 10 with memory 11 and processor 12 is shown. It will be understood by those skilled in the art that... Figure 1 The structure shown does not constitute a limitation on the electronic device 10, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0081] The memory 11 in the electronic device 10 stores multiple computer-readable instructions to implement an image data generation method. The processor 12 can execute multiple instructions to achieve: acquiring a three-dimensional model of the target object, a real defect image, and a normal image, wherein the real defect image refers to a real-life image of the target object with defects, and the normal image refers to a real-life image of the target object without defects; generating a synthetic defect image and its annotation information based on the three-dimensional model and the real defect image, wherein the annotation information of the synthetic defect image includes the distribution information of the defects in the three-dimensional model in the synthetic defect image; performing simulation processing on the normal image of the target object based on a preset generative adversarial network model to generate a simulated defect image; and constructing an image fusion dataset based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image in the multiple synthetic defect images.

[0082] The memory 11 in the electronic device 10 stores multiple computer-readable instructions to implement an anomaly detection method. The processor 12 can execute multiple instructions to achieve: constructing an image fusion dataset based on an image data generation method; training a preset anomaly detection model based on the image fusion dataset to obtain a trained anomaly detection model; and performing anomaly detection on a target object based on the trained anomaly detection model.

[0083] Specifically, the processor 12's implementation method for the above instructions can be found in [reference needed]. Figure 2 , Figure 3 , Figure 4 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0084] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 10 and does not constitute a limitation on the electronic device 10. The electronic device 10 can be a bus topology or a star topology. The electronic device 10 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, the electronic device 10 may also include input / output devices, network access devices, etc.

[0085] It should be noted that electronic device 10 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0086] The memory 11 includes at least one type of computer-readable storage medium, which can be non-volatile or volatile. Computer-readable storage media include flash memory, portable hard drives, multimedia cards, card-type memories (e.g., SD memory, DX memory, etc.), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 10, such as a portable hard drive of the electronic device 10. In other embodiments, the memory 11 can also be an external storage device of the electronic device 10, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 10. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 10, such as the code of an image data generation or anomaly detection program, but also to temporarily store data that has been output or will be output.

[0087] In some embodiments, the processor 12 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 12 is the control unit of the electronic device 10, connecting various components of the electronic device 10 via various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing an image data generation or anomaly detection program) and calls data stored in the memory 11 to perform various functions and process data for the electronic device 10.

[0088] The processor 12 executes the operating system of the electronic device 10 and various installed applications. The processor 12 executes the applications to implement the steps in each of the above-described embodiments of the image data generation or anomaly detection method.

[0089] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in memory 11 and executed by processor 12 to complete this application. One or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in electronic device 10.

[0090] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. The software functional module stored in the storage medium includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor 12 to execute a portion of an image data generation or anomaly detection method according to various embodiments of this application.

[0091] If the modules / units integrated in the electronic device 10 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 12, it can implement the steps of the various method embodiments described above.

[0092] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory, and other types of memory.

[0093] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0094] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 1 The symbol is represented by only one arrow, but this does not mean that there is only one bus or one type of bus. The bus is configured to implement communication between memory 11 and at least one processor 12, etc.

[0095] This application also provides a computer-readable storage medium (not shown), which stores computer-readable instructions. These computer-readable instructions are executed by a processor 12 in an electronic device 10 to implement an image data generation or anomaly detection method according to any of the above embodiments.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0097] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0099] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A method for generating image data, characterized in that, The method includes: Acquire a 3D model of the target object, a real defect image, and a normal image, wherein the real defect image refers to a real-shot image of the target object with defects, and the normal image refers to a real-shot image of the target object without defects. Based on the 3D model and the real defect image, a synthetic defect image and the annotation information of the synthetic defect image are generated, wherein the annotation information of the synthetic defect image includes the distribution information of the defects of the 3D model in the synthetic defect image; Based on a preset adversarial generative network model, the normal image of the target object is simulated to generate multiple simulated defect images. An image fusion dataset is constructed based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image among the multiple synthetic defect images.

2. The image data generation method as described in claim 1, characterized in that, The process of generating a synthetic defect image and its annotation information based on the 3D model and the real defect image includes: A defect texture map is generated based on the real defect image; The defect texture map is mapped onto the surface of the 3D model to obtain a simulated 3D model; The simulated 3D model is rendered using rendering technology to generate the synthetic defect image and its annotation information.

3. The image data generation method as described in claim 2, characterized in that, The step of mapping the defect texture map onto the surface of the 3D model to obtain the simulation 3D model includes: Map the surface of the three-dimensional model onto a two-dimensional plane; The defect texture map is seamlessly edited to generate a seamless defect texture map; The seamless defect texture map is mapped onto the two-dimensional plane to obtain the simulated three-dimensional model.

4. The image data generation method as described in claim 3, characterized in that, The method further includes, after mapping the seamless defect texture map onto the two-dimensional plane: The layout information of the seamless defect texture map on the surface of the 3D model is adjusted based on preset adjustment parameters.

5. The image data generation method as described in claim 2, characterized in that, The rendering of the simulated 3D model based on rendering technology to generate the synthetic defect image and the annotation information of the synthetic defect image includes: Obtain the rendering parameters of the simulated 3D model; The simulated 3D model is rendered based on the rendering engine and the rendering parameters to generate the synthetic defect image, the identifier of each defect in the simulated 3D model, and the relevant information of each defect. The annotation information of the synthetic defect image is generated based on the identifier of each defect and the relevant information of each defect.

6. The image data generation method as described in claim 2, characterized in that, Before generating a defect texture map based on the real defect image, the method further includes: A preset scene texture map is mapped onto the surface of the 3D model, and the scene texture map includes a variety of different texture attributes.

7. The image data generation method as described in claim 1, characterized in that, Before constructing the image fusion dataset based on multiple real defect images, multiple synthetic defect images, multiple simulated defect images, and the annotation information of each synthetic defect image among the multiple synthetic defect images, the method further includes: Based on the adversarial generative network model, the style of the synthetic defect image is transformed into a style similar to that of the real defect image to obtain the transformed synthetic defect image.

8. An anomaly detection method, characterized in that, The method includes: An image fusion dataset is constructed based on the image data generation method as described in any one of claims 1-7; The preset anomaly detection model is trained based on the image fusion dataset to obtain the trained anomaly detection model. Anomaly detection is performed on the target object based on the trained anomaly detection model.

9. The anomaly detection method as described in claim 8, characterized in that, The step of training a preset anomaly detection model based on the image fusion dataset to obtain a trained anomaly detection model includes: The preset anomaly detection model is pre-trained based on the multiple simulated defect images to obtain a first detection model; The first detection model is fine-tuned and trained based on the multiple synthetic defect images and the annotation information of each synthetic defect image to obtain the second detection model. The second detection model is fine-tuned and trained based on the real defect image to obtain the trained anomaly detection model.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the image data generation method as described in any one of claims 1 to 7 or the anomaly detection method as described in any one of claims 8 to 9.

11. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the image data generation method as described in any one of claims 1 to 7 or the anomaly detection method as described in any one of claims 8 to 9.

Images