An abnormal sample generation method, device, equipment and medium

Abstract

The application discloses an abnormal sample generation method, device and equipment and medium, and belongs to the technical field of defect detection. The method comprises the following steps: extracting a target region of a target image to which a foreign matter is to be added, scaling the target region, aligning the scaled target region with a foreign matter generation text, obtaining a foreign matter addition region, fusing a foreign matter addition image with the target image, and obtaining the foreign matter addition image; a digital twin model corresponding to a pantograph is built, and the digital twin model is modified by using Blender to obtain a fault foreground image; the fault foreground image is fused with a target background image to obtain a fault addition image; and the foreign matter addition image and the fault addition image are both determined as abnormal samples of the pantograph. By using the method, the abnormal samples of the pantograph have the generation ability of zero samples, the detection requirements of an intelligent inspection system can be met, and the generated abnormal samples will not have problems such as structural distortion and abnormal device connection relationship.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method, apparatus, equipment and medium for generating abnormal samples. Background Technology

[0002] The pantograph is a critical component of the high-voltage system of a high-speed train. Under the high-speed operating conditions of the train, the pantograph is prone to foreign object adhesion and internal component malfunctions. If these situations are not detected and addressed in time, they may lead to pantograph-catenary accidents, seriously affecting the safety and stability of the high-speed train during operation.

[0003] Currently, computer vision-based intelligent inspection systems are commonly used to monitor the operational status of pantographs. However, these systems primarily rely on deep learning-based anomaly detection models, whose results are highly dependent on the diversity and coverage of the training samples. Anomalies in pantographs are typically low-probability events with a long-tailed distribution. Due to the difficulty in collecting samples of foreign objects attached to the pantograph under varying weather conditions, and because internal component failures often indicate an accident, such failures are rarely captured during routine inspections. Adding foreign objects or damaging internal components to the pantograph to collect anomaly samples poses extremely high safety risks, resulting in a severe scarcity of anomaly samples and significantly impacting the accuracy of intelligent inspection systems in detecting pantograph operational status.

[0004] In existing technologies, to address this technical problem, GANs (Generative Adversarial Networks) or DDPMs (Denoising Diffusion Probabilistic Models) are typically used to generate anomaly samples from pantographs. However, these models rely on specific categories of anomaly data to generate a certain number of anomaly samples, lacking the ability to generate zero-sample data. Furthermore, the resolution of anomaly samples generated by these models is limited, typically within 512×512 pixels, which is insufficient to meet the detection requirements of intelligent inspection systems. Additionally, the anomaly samples generated by these models lack an understanding of the physical structure of the pantograph, often resulting in structural distortion and abnormal device connections. Currently, there is no effective solution to this technical problem.

[0005] Therefore, how to provide a better method for generating abnormal samples to avoid the above-mentioned problems is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for generating abnormal samples, so that the abnormal samples of the pantograph have the ability to generate zero samples, and can meet the detection requirements of the intelligent inspection system. Furthermore, it ensures that the generated abnormal samples do not exhibit structural distortion or abnormal device connection relationships. The specific solution is as follows: To address the aforementioned technical problems, this invention provides a method for generating abnormal samples, comprising: Acquire the target image; the target image is any image from the original image set of the pantograph; Extract the target region of the target image from which the foreign object is to be added, and scale the target region according to a preset standard format to obtain the target scaled region; Align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and then fuse the foreign object addition area with the target image to obtain the foreign object addition image; A kinematic simulation algorithm based on skeletal binding is used to build a digital twin model corresponding to the pantograph, and Blender is used to modify the digital twin model to simulate the fault that occurs in the pantograph and obtain a fault foreground image. The fault foreground image is fused with the target background image to obtain a fault-added image; the target background image is the background image in the target image. Both the foreign object image and the fault image are identified as abnormal samples of the pantograph.

[0007] Preferably, extracting the target region in the target image from which the foreign object is to be added includes: Mark the target area in the target image where the foreign object is to be added; Based on the proportional relationship between the target region and the target image, the target region is extracted using an adaptive interpolation algorithm.

[0008] Preferably, marking the target region in the target image where the foreign object is to be added includes: The target area in the target image to which the foreign object is to be added is marked according to the user's instructions.

[0009] Preferably, scaling the target area according to a preset standard format to obtain a target scaled area includes: The target scaled region is obtained by scaling the target region according to the preset standard format based on the Lanczos interpolation algorithm.

[0010] Preferred options also include: If there is a white border area in the target scaling area, then the white border area is filled.

[0011] Preferably, aligning the target scaling area with the foreign object generation text to obtain the foreign object addition area includes: The target scaling region is aligned with the foreign object generated text using the attention cross-interaction algorithm in Transformer to obtain the foreign object addition region.

[0012] Preferred options also include: The attention crossover algorithm is optimized using SVD.

[0013] Preferably, the step of modifying the digital twin model using Blender to simulate a fault in the pantograph and obtaining a fault foreground image includes: Based on user instructions, the settings parameters in the digital twin model are modified using the Blender to simulate the fault that occurs in the pantograph, and the fault foreground image is obtained.

[0014] Preferably, the step of modifying the digital twin model using Blender to simulate a fault in the pantograph and obtaining a fault foreground image includes: Based on user instructions, the specified device in the digital twin model is bent and deformed using the Blender to simulate the fault of the pantograph and obtain the fault foreground image.

[0015] Preferred options also include: The fault foreground image was optimized using the PBR material system.

[0016] Preferably, the step of fusing the fault foreground image with the target background image to obtain the fault-added image includes: The fault foreground image and the target background image are fused using the Alpha blending node in the synthesizer to obtain the fault-added image; the synthesizer is a component of the Blender.

[0017] Preferred options also include: Gaussian noise is added to the fault foreground image based on the noise distribution pattern of the target background image.

[0018] Preferred options also include: Remove non-environmental interference features from the target image to obtain the first processed image; Extract the pantograph support structure from the first processed image and remove the support structure from the first processed image to obtain the second processed image; The pantograph base and connecting rod are removed from the second processed image to obtain the target background image.

[0019] Preferred options also include: A global residual scan is performed on the target background image to remove artifacts from the target background image.

[0020] Preferred options also include: Optical distortion correction is performed on the image added to the faulty image.

[0021] Preferably, acquiring the target image includes: Acquire original image sets of the pantograph under different operating environments; the different operating environments of the pantograph include: tunnel operating environment, maintenance depot operating environment and open-air operating environment; The original image set of the pantograph under the target operating environment is filtered, and one image is selected from the original image set of the pantograph under the target operating environment to obtain the target selected image; the target operating environment is any operating environment of the pantograph under different operating environments.

[0022] Preferred options also include: The background image of the target screening image is determined by using histogram matching parameters and noise distribution model, thus obtaining the target screening background, and the target screening background is determined as the background image of the pantograph in the target operating environment.

[0023] To address the aforementioned technical problems, the present invention also provides an apparatus for generating abnormal samples, comprising: The image acquisition module is used to acquire a target image; the target image is any image from the original image set of the pantograph. The region extraction module is used to extract the target region of the target image to which the foreign object is to be added, and to scale the target region according to a preset standard format to obtain the target scaled region; The foreign object addition module is used to align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and to fuse the foreign object addition area with the target image to obtain the foreign object addition image; The fault generation module is used to build a digital twin model corresponding to the pantograph based on the kinematic simulation algorithm of skeleton binding, and to modify the digital twin model using Blender to simulate the faults that occur in the pantograph and obtain a fault foreground image. An image fusion module is used to fuse the fault foreground image with the target background image to obtain a fault-added image; the target background image is the background image in the target image; The sample determination module is used to determine both the foreign object added image and the fault added image as abnormal samples of the pantograph.

[0024] To address the aforementioned technical problems, the present invention also provides an apparatus for generating abnormal samples, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of a method for generating an anomalous sample as disclosed above.

[0025] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an abnormal sample generation method as disclosed above.

[0026] Beneficial Effects: This invention generates two types of anomaly samples related to the pantograph: one is an image of foreign object attachment to the pantograph, and the other is an image of malfunction attachment to the pantograph. When generating the image of foreign object attachment, the target area of ​​the foreign object to be added is first extracted from the target image, and then scaled according to a preset standard format to obtain a scaled target area. Next, the scaled target area is aligned with the foreign object generation text to obtain the foreign object attachment area, and then the foreign object attachment area is fused with the target image to obtain the foreign object attachment image. In this configuration, the foreign object attachment image is essentially generated as different types of foreign object attachment samples in a specified area of ​​the pantograph based on the foreign object generation text. This ensures the controllability and diversity of the location of the added foreign object in the foreign object attachment image. When generating fault-added images for the pantograph, the first step is to build a digital twin model corresponding to the pantograph using a kinematic simulation algorithm based on skeletal binding. Blender is then used to modify the digital twin model, simulating faults in the pantograph and obtaining a fault foreground image. This foreground image is then fused with the target background image to obtain the fault-added image. During the generation of the fault-added image, because the foreground image is generated based on the pantograph's mechanical structure, component connections, and kinematic constraints in the digital twin model, and the target background image is the background image from the original pantograph image, the fault-added image generated from the foreground and target background images ensures that there will be no structural distortion or abnormal component connections.

[0027] Furthermore, since the generated foreign object and fault images do not rely on the anomaly samples corresponding to the pantograph itself, this anomaly sample generation method possesses zero-sample generation capability. Simultaneously, because the foreign object and fault images generated by this method are based on the fusion of the foreign object or fault with the original pantograph image to generate anomaly samples, the resolution of the original pantograph image is essentially not damaged. Therefore, this method allows anomaly samples to have high image resolution, meeting the detection requirements of intelligent inspection systems.

[0028] Correspondingly, the device, equipment, and medium for generating abnormal samples provided by the present invention also have the above-mentioned beneficial effects. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0030] Figure 1 A flowchart illustrating a method for generating abnormal samples according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an abnormal sample generation system provided in an embodiment of the present invention; Figure 3 A flowchart illustrating the process of adding images to generate foreign objects; Figure 4 A flowchart illustrating the process of adding images to generate faults; Figure 5 This is a structural diagram of an abnormal sample generation device provided in an embodiment of the present invention; Figure 6 This is a structural diagram of an abnormal sample generation device provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] To enable those skilled in the art to more clearly understand the implementation principle of this invention, a brief introduction to the relevant background technology involved in this application is provided below. The pantograph is a key component of the high-voltage system of a high-speed train, typically installed on the roof of the train and exposed to the elements for extended periods in an extremely harsh working environment. Under high-speed operating conditions, even the smallest foreign object adhering to the pantograph (e.g., plastic bags, bird feathers, agricultural mulch film, etc.) can disrupt the aerodynamic balance of the pantograph, leading to pantograph disconnection or sparking. Furthermore, under conditions of high-frequency vibration, strong airflow impact, and intense friction between the pantograph and the catenary, the carbon plate, pantograph angle, spring box, and metal support of the pantograph are highly susceptible to fatigue fracture. Moreover, sometimes the pantograph also experiences structural failures such as impact deformation, material chipping, loose fasteners, and missing components. Currently, the main method for detecting the pantograph's operating status is through intelligent inspection systems based on computer vision (such as vehicle-mounted overhead contact line operation status detection devices, referred to as 3C devices). However, abnormal pantograph samples are typical long-tailed, low-probability events, and the extreme scarcity of abnormal samples severely restricts the detection accuracy of intelligent inspection systems.

[0033] This application provides a method for generating abnormal samples. This method not only enables the generation of zero-sample abnormal samples from pantographs but also meets the detection requirements of intelligent inspection systems. Furthermore, it ensures that the generated abnormal samples do not exhibit structural distortion or abnormal device connections. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 A flowchart of an abnormal sample generation method provided in an embodiment of the present invention is shown. The method includes: Step S11: Obtain the target image; the target image is any image from the original image set of the pantograph; Step S12: Extract the target area of ​​the object to be added from the target image, and scale the target area according to the preset standard format to obtain the target scaled area; Step S13: Align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and fuse the foreign object addition area with the target image to obtain the foreign object addition image; Step S14: Based on the kinematic simulation algorithm of skeleton binding, build a digital twin model corresponding to the pantograph, and use Blender to modify the digital twin model to simulate the fault of the pantograph and obtain the fault foreground image. Step S15: Fuse the fault foreground image with the target background image to obtain the fault-added image; the target background image is the background image in the target image; Step S16: Both the foreign object image and the fault image are identified as abnormal samples of the pantograph.

[0034] It should be noted that the execution entity of the abnormal sample generation method described in this embodiment can be a server or a terminal device with image processing and 3D rendering capabilities; no specific limitation is made here. Furthermore, in this embodiment, the target image refers to any image in the original pantograph image set. The original pantograph image set can be a collection of pantograph images collected during the actual operation of the EMU by image acquisition devices such as onboard catenary operation status detection devices, or a collection of pantograph images collected in scenarios such as maintenance depots; the source of the original pantograph image set is not specifically limited here.

[0035] In practical applications, the generation of pantograph foreign object anomaly samples is primarily driven by a multimodal image editing model; that is, the foreign object addition image is generated by the multimodal image editing model. When acquiring the foreign object addition image, the target region for adding the foreign object must first be identified from the target image. Specifically, the target region for adding foreign objects such as feathers, strips of cloth, or plastic bags needs to be identified from the target image. Since the size of the target region extracted from the target image is irregular and cannot well fit the input conditions of the multimodal image editing model, it needs to be scaled according to a preset standard format to obtain the scaled target region. The preset standard format can be set according to the input requirements of the model used; no specific limitation is made here.

[0036] It should be noted that the multimodal image editing model can be set as Qwen-Image-Edit. Because Qwen-Image-Edit has general semantic understanding capabilities, it does not require training or fine-tuning for specific categories of foreign objects. Users only need to describe the object in natural language, and Qwen-Image-Edit can directly generate foreign objects of various shapes and realistic textures based on its powerful common sense reasoning ability, even without having seen a specific sample of that type of foreign object before.

[0037] Next, the target scaling region is aligned with the foreign object generation text. The purpose of this step is to establish a correspondence between the text information describing the foreign object to be generated and the image information of the target scaling region. This allows the multimodal image editing model to generate a foreign object in the target scaling region that is semantically consistent with the foreign object generation text, thus obtaining the foreign object addition region. For example, if you want to add a plastic bag to the target scaling region, then the foreign object generation text corresponding to the target scaling region would be: Add a plastic bag to the target scaling region.

[0038] Finally, the foreign object-added area is fused back into the target image to obtain the foreign object-added image, thus creating an abnormal sample with foreign objects added to the original pantograph image. In this configuration, since the foreign object's generation location is defined by the target area and its type and form are defined by the foreign object generation text, the method described in this invention can precisely specify the foreign object's generation location and type and form on the original pantograph image. This ensures the controllability and diversity of the foreign object's addition location on the original pantograph image.

[0039] In practical applications, 3D software is responsible for generating anomaly samples of the pantograph structure; that is, 3D software generates fault-adding images. During the generation of these images, the target image (i.e., the actual image of the pantograph) is used as a visual reference to build a corresponding digital twin model of the pantograph, and a complete rigid skeleton system is constructed for this digital twin model.

[0040] In other words, the digital twin model incorporates the mechanical motion constraints of the pantograph. With this setup, when the user needs to adjust the pantograph's attitude in the 3D software, there's no need to individually adjust the angle of each link within the pantograph. Only the core lifting control component needs adjustment, and the digital twin model can automatically calculate the linkage angles of each joint in the pantograph. This allows for a realistic simulation of any state of the pantograph from the lowered to the raised position. Based on this posture simulation of the pantograph's physical structure, the digital twin model can adjust the pantograph's angle and height in a manner consistent with mechanical principles, thereby achieving a high degree of overlap and pose matching between the digital twin model and the pantograph's attitude in the target image.

[0041] When acquiring the foreground image of the malfunction, a highly flexible editing workflow based on geometric nodes and modifiers was built within the Blender 3D editing environment. Specifically, after loading the digital twin model into Blender, users can directly edit the digital twin model through Blender's internal tool panels.

[0042] If a user wants to control and modify a specific location of the pantograph in the digital twin model, the user can directly adjust the values ​​at the input end of the geometric node or drag the auxiliary geometry (such as a Boolean cut body). Then, the user can preview the location in real time in Blender's 3D view and accurately locate the position of the pantograph's break point in the digital twin model.

[0043] If a user wants to control the degree of damage to a specific component in the pantograph within the digital twin model, they can manually adjust the relevant parameter settings based on the visual effect of the digital twin model in Blender until the surface roughness and defect area of ​​the component meet expectations.

[0044] Understanding this, building a digital twin model of the actual pantograph is essentially placing that digital twin model within the 3D simulation software Blender. In this scenario, users can freely adjust or modify relevant settings within Blender based on anticipated faults (such as component breakage, component wear, loose screws, pantograph tilt, etc.) to simulate pantograph malfunctions under real-world conditions. This process simulates the pantograph's faults, resulting in a fault foreground image. The fault foreground image refers to the 3D model containing the pantograph fault type that needs to be overlaid on the target background image.

[0045] When acquiring the target background image, the background image can be obtained simply by extracting it from the target image. In other words, the actual working environment image remaining after removing the main structure of the pantograph from the target image is the target background image.

[0046] After obtaining the fault foreground image and the target background image, fusing them together yields the fault-added image. It's conceivable that because the target background image retains the true lighting distribution and environmental details of the target image, fusing the fault foreground image with the target background image can significantly improve the realism and environmental adaptability of the fault-added image.

[0047] Once the images of foreign objects and faults are acquired, both can be identified as abnormal samples of the pantograph. These abnormal samples can then be used to train the abnormal sample detection model in the intelligent inspection system, thereby improving the detection accuracy of the intelligent inspection system for detecting foreign object attachments and structural faults on the pantograph.

[0048] In summary, this embodiment generates two types of abnormal samples related to the pantograph: one is a foreign object addition image of the pantograph, and the other is a fault addition image of the pantograph. When generating the foreign object addition image, the target area of ​​the foreign object to be added is first extracted from the target image, and then scaled according to a preset standard format to obtain a scaled target area. Next, the scaled target area is aligned with the foreign object generation text to obtain the foreign object addition area, and then the foreign object addition area is fused with the target image to obtain the foreign object addition image. Under this setup, the foreign object addition image is essentially generated as different types of foreign object attachment samples in a specified area of ​​the pantograph based on the foreign object generation text. This ensures the controllability and diversity of the generation location of the added foreign object in the foreign object addition image. When generating fault-added images for the pantograph, the first step is to build a digital twin model corresponding to the pantograph using a kinematic simulation algorithm based on skeletal binding. Blender is then used to modify the digital twin model, simulating faults in the pantograph and obtaining a fault foreground image. This foreground image is then fused with the target background image to obtain the fault-added image. During the generation of the fault-added image, because the foreground image is generated based on the pantograph's mechanical structure, component connections, and kinematic constraints in the digital twin model, and the target background image is the background image from the original pantograph image, the fault-added image generated from the foreground and target background images ensures that there will be no structural distortion or abnormal component connections.

[0049] Furthermore, since the generated foreign object and fault images do not rely on the anomaly samples corresponding to the pantograph itself, this anomaly sample generation method possesses zero-sample generation capability. Simultaneously, because the foreign object and fault images generated by this method are based on the fusion of the foreign object or fault with the original pantograph image to generate anomaly samples, the resolution of the original pantograph image is essentially not damaged. Therefore, this method allows anomaly samples to have high image resolution, meeting the detection requirements of intelligent inspection systems.

[0050] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above step of extracting the target area in the target image where the foreign object is to be added includes: Mark the target area in the target image where the foreign object will be added; The target region is extracted based on the proportional relationship between the target region and the target image, using an adaptive interpolation algorithm.

[0051] In this embodiment, when extracting the target region, the target region in the target image to which the foreign object is to be added is first marked, and the minimum bounding rectangle is calculated based on the region marked by the user. Then, the minimum bounding rectangle is expanded in context so that the minimum bounding rectangle can contain the necessary environmental features.

[0052] It should be noted that when extracting pixel data of the target region from a high-resolution target image, it is necessary to extract the target region according to the ratio (i.e., scaling relationship) of the target region relative to the target image and to use an adaptive interpolation strategy.

[0053] Specifically, when it is necessary to reduce the target area, the bilinear interpolation algorithm can be used because it has low computational cost and can effectively suppress high-frequency aliasing noise, preventing moiré patterns during target area extraction. When it is necessary to enlarge the target area, the bicubic interpolation algorithm can be used, and the weighted calculation values ​​of the 16 pixels surrounding the target area can be used to ensure a smooth transition of the extracted target area edges, thereby avoiding jagged edges in the extracted target area.

[0054] Obviously, the technical solution provided in this embodiment can better extract the target area from the target image when a foreign object is to be added to it.

[0055] As a preferred embodiment, the above step of marking the target area in the target image where the foreign object is to be added includes: Mark the target area in the target image where the foreign object will be added, according to the user's instructions.

[0056] In this embodiment, the target area where foreign objects are to be added to the target image is marked according to user instructions. That is, the user can start a manual interactive mask drawing process and use the brush tool on the operation interface of the multimodal image editing model to paint over the target area in the target image where foreign objects are to be added, and mark the target area according to the user's painting results.

[0057] Compared to automatic segmentation of the target region, manually segmenting and labeling the target region allows for a more precise definition of the spatial boundaries of foreign objects in the target image, completely avoiding the offset of the generated position of foreign objects in the target region caused by false detections in the segmentation algorithm. Furthermore, this method provides clearer guidance for the generation region of multimodal image editing models, enabling them to automatically deduce the physically logical shape and posture of foreign objects within the target region.

[0058] Obviously, the technical solution provided in this embodiment can more accurately control the generation position of foreign objects on the target image and ensure the accuracy of the generation position of foreign objects in the image with added foreign objects.

[0059] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above step of scaling the target area according to a preset standard format to obtain the target scaled area includes: Based on the Lanczos interpolation algorithm, the target area is scaled according to a preset standard format to obtain the target scaled area.

[0060] Understandably, because the extracted target region has irregular dimensions, it cannot well adapt to the input conditions of the multimodal image editing model. Therefore, in this embodiment, the Lanczos interpolation algorithm is used to normalize and scale the target region to obtain the scaled target region. Since the filter kernel in the Lanczos interpolation algorithm has excellent sidelobe suppression capability in the frequency domain, when using the Lanczos interpolation algorithm to scale the target region, image details in the target region can be preserved to the greatest extent.

[0061] The expression for the Lanczos interpolation algorithm is shown in Equation 1: Formula 1: ; In the formula, This is the filter kernel radius parameter (usually taken as 2 or 3). .

[0062] Obviously, the technical solution provided in this embodiment can further ensure the image clarity of the target scaling area.

[0063] In a preferred embodiment, the above method further includes: If there are white borders in the target scaling area, then fill the white borders.

[0064] In this embodiment, considering that pixel offset may occur due to image stitching when cropping and scaling the target area, a white border area will be formed at the edge of the target scaling area.

[0065] In practical applications, if white borders exist within the target scaling area, these borders need to be filled. Specifically, the white borders can be filled using the pixel values ​​surrounding them. It's conceivable that by filling the white borders, the problems of missing or offset edge pixels that occur when scaling images using conventional methods can be largely eliminated.

[0066] Obviously, the technical solution provided in this embodiment can avoid the influence of the white border area on the foreign object generation effect.

[0067] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above steps: aligning the target scaling area with the foreign object generation text to obtain the foreign object addition area, include: The attention cross-interaction algorithm in Transformer is used to align the target scaling region with the text generated by the foreign object, thus obtaining the foreign object addition region.

[0068] In this embodiment, the foreign object generation text refers to natural language instructions used to describe the foreign object to be generated, such as: add a plastic bag, strip of cloth, feather, etc. to the target area. After obtaining the target scaling area, the Cross-Attention algorithm in the Transformer architecture can be used to establish the correspondence between the textual semantic features of the foreign object generation text and the image visual features of the target scaling area, and generate a foreign object in the target scaling area that is semantically consistent with the foreign object generation text, thereby obtaining the foreign object addition area.

[0069] Specifically, in practical applications, Formula 2 can be used to calculate the weight matrix in the attention cross algorithm, where the expression of Formula 2 is: Formula 2: ; In the formula, and Indicates that foreign objects generate text. This means encoding the image features corresponding to the text generated by the foreign object into a query. Indicates the feature dimension.

[0070] By using the attention cross-processing algorithm, the understanding capabilities of the multimodal image editing model can be fully utilized, allowing the model to automatically query the semantic information of the target background image and automatically adjust the brightness values ​​of the generated pixels according to the illumination distribution of the target background image. This enables a more natural image fusion effect.

[0071] Obviously, the technical solution provided in this embodiment can accurately and reliably align the target scaling area with the text generated from the foreign object.

[0072] In a preferred embodiment, the above method further includes: The attention crossover algorithm is optimized using SVD.

[0073] In practical applications, multimodal image editing models have a huge number of parameters, high memory consumption, and slow inference speed. To address this technical problem, this embodiment utilizes Nunchaku's SVD (Singular Value Decomposition) to optimize the attention crossover algorithm.

[0074] Specifically, SVD quantization technology is used to decompose and compress the large-scale weight matrix in the attention cross algorithm into a low-rank form, and the amount of video memory used can be reduced by 40% to 50% while maintaining almost no loss in the quality of the generated image.

[0075] In addition, customized GPU (Graphics Processing Unit) computing cores can be used to further optimize the computational efficiency of the Attention operator in the Transformer architecture, thereby further improving the image generation speed and image editing response time.

[0076] Clearly, the technical solution provided in this embodiment enables the multimodal image editing model to run more smoothly on a single graphics card and to be deployed locally on a server with lower design costs.

[0077] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above steps include: modifying the digital twin model using Blender to simulate a fault in the pantograph and obtaining a fault foreground image, including: Based on user instructions, the settings parameters in the digital twin model are modified using Blender to simulate the fault in the pantograph and obtain the fault foreground image. Alternatively, based on user instructions, Blender can be used to bend and deform specified components in the digital twin model to simulate a fault in the pantograph and obtain a fault foreground image.

[0078] In this embodiment, since the digital twin model has built-in mechanical motion constraint logic for the pantograph, the user does not need to adjust the angle of each link individually. By simply modifying the setting parameters of the core lifting control components, the digital twin model can automatically calculate the linkage angle of each joint in the pantograph and thus simulate any state of the pantograph from the lowered position to the raised position in real-world conditions.

[0079] In addition, users can modify the settings parameters at the geometric node input end (such as noise scaling and cutting volume values) to simulate faults such as component breakage, wear, loose screws, and bow angle tilt, and quantify and control the location and degree of damage.

[0080] In practical applications, users can also manually bend and deform specified devices in the digital twin model in Blender to simulate the faults that occur in the pantograph and obtain a fault foreground image.

[0081] For example, to simulate the bending deformation of a pantograph after being impacted by an external force, users can use two methods. One method is to manually adjust the angle and axis of the bend modifier; the other is to directly invoke the interactive bend tool in Blender's edit mode using a keyboard shortcut and perform real-time nonlinear deformation on the mesh vertices. In both methods, users can preview the pantograph's deformation effect in real time on the Blender interface. Furthermore, in both methods, users can precisely modify and adjust the degree of bending of the pantograph.

[0082] Clearly, the technical solution provided in this embodiment can more accurately and reliably simulate the faults that occur in the pantograph.

[0083] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above method further includes: The PBR material system was used to optimize the foreground image of the fault.

[0084] To address the issue of realism in 3D rendering, this embodiment also employs a high-precision PBR (Physically Based Rendering) material system to optimize the faulty foreground image.

[0085] Specifically, Blender pre-sets digital materials that are highly consistent with the actual pantograph components to accurately simulate the gloss, reflectivity, and matte texture of the pantograph's metal surface and carbon skateboard. For example, at the simulated cross-section of the pantograph, the PBR material system automatically applies fresh metal or carbon materials to the cross-section to reflect the textural differences between the cross-section and the surrounding structure.

[0086] Furthermore, the PBR material system includes a library of lighting presets for different scene environments. This means that even when the pantograph operates in various weather conditions (e.g., daytime, nighttime, sunny, and rainy), it can still find the corresponding ambient light map from the library. After selecting the ambient light map, the PBR material also supports dynamic adjustment of the light intensity. For example, in sunny mode, the light intensity can be increased to create sharp shadows in the foreground image of the fault; in cloudy or early morning mode, the light intensity can be reduced and the color temperature adjusted to increase diffuse reflection in the foreground image of the fault; and in nighttime or tunnel mode, the ambient light intensity can be reduced and artificial point light sources can be activated to simulate the high-contrast lighting effect of searchlights in the foreground image of the fault.

[0087] Obviously, the technical solution provided in this embodiment can not only improve the image quality of the fault foreground image, but also enhance the realism of the fault foreground image.

[0088] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above step of fusing the fault foreground image and the target background image to obtain a fault-added image includes: The fault foreground image and the target background image are fused using the Alpha blending node in the compositor to obtain the fault-added image; the compositor is a component of Blender.

[0089] In this embodiment, when fusing the fault foreground image and the target background image, the target background image is used as a background node, and the rendered fault foreground image is used as a foreground layer. Then, the fault foreground image and the target background image are fused through the Alpha blending node in the compositor to obtain the fault-added image. This method not only ensures that the fault foreground image can cover the target background image with the correct foreground layer, but also automatically replaces the background layer in the fault-added image and minimizes the interference of lighting on the background image.

[0090] Obviously, the technical solution provided in this embodiment can ensure the correct hierarchical relationship when the faulty foreground image is fused with the target background image.

[0091] In a preferred embodiment, the above method further includes: Gaussian noise is added to the fault foreground image based on the noise distribution pattern of the target background image.

[0092] In practical applications, the rendered faulty foreground image differs from the actual captured target background image in pixel statistical characteristics, easily leaving traces of artificial synthesis in the faulty foreground image. In this embodiment, to avoid the above situation, the noise distribution pattern of the target background image is analyzed, and Gaussian noise matching the noise distribution pattern of the target background image is added to the faulty foreground image. This ensures the consistency of image quality between the faulty foreground image and the target background image.

[0093] Clearly, the technical solution provided in this embodiment can eliminate artificial synthesis traces in faulty foreground images.

[0094] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above method further includes: Remove non-environmental interference features from the target image to obtain the first processed image; Extract the pantograph support structure from the first processed image and remove the support structure from the first processed image to obtain the second processed image; The pantograph base and connecting rod are removed from the second processed image to obtain the target background image.

[0095] In this embodiment, to simulate the actual working environment of the pantograph, a synthesis strategy based on the actual background of the pantograph is used during the fusion of the fault foreground image and the target background image. For the core step of target background image extraction, this embodiment proposes a three-stage cascaded image element removal method based on a multimodal visual large model. This method designs a step-by-step stripping workflow of "character-body-base".

[0096] In other words: First, non-environmental interference elements such as characters and watermarks in the target image are identified and erased to obtain the first processed image; then, the main support structure of the pantograph in the first processed image is precisely segmented and removed to obtain the second processed image; finally, the base and connecting rod of the pantograph remaining in the second processed image are removed to obtain the target background image. This layered and progressive removal strategy can preserve the light distribution and environmental details of the pantograph under real working conditions to the greatest extent and achieve lossless extraction of complex background information.

[0097] Obviously, the target background image can be extracted from the target image without loss through the technical solution provided in this embodiment.

[0098] In a preferred embodiment, the above method further includes: A global residual scan is performed on the target background image to remove artifacts from the target background image.

[0099] To ensure the quality of the extracted target background image, in this embodiment, after obtaining the target background image, a global residual scan is performed on the target background image to identify and remove residual artifacts in the target background image, thereby further improving the image quality of the target background image.

[0100] Clearly, the technical solution provided in this embodiment can avoid the impact of residual artifacts on the realism of the fault-adding image.

[0101] In a preferred embodiment, the above method further includes: Add optical distortion correction to the faulty image.

[0102] In this embodiment, in order to further reduce the difference between the synthesized image and the image captured by the vehicle-mounted inspection camera, an optical distortion simulation algorithm is introduced and used to correct the optical distortion of the fault-added image.

[0103] Specifically, a polynomial radial distortion simulation algorithm based on the Brown-Conrady model can be used to accurately simulate the bending path of light in a wide-angle lens group, thereby reproducing the chromatic aberration phenomenon commonly seen at the edges of images captured by real industrial lenses.

[0104] In practical applications, Formula 3 can be used to perform nonlinear geometric transformations on fault-adding images, where the expression of Formula 3 is: Formula 3: ; In the formula, Radial distance, , and All of these are radial distortion coefficients. By adjusting the various parameters in Formula 3, the bending path of light in a wide-angle lens group can be accurately simulated.

[0105] Furthermore, spectral dispersion simulation can be integrated into the polynomial radial distortion simulation algorithm, and subtle differences can be set for the red, green, and blue channels. This process aims to reproduce the chromatic aberration commonly seen at the edges of real industrial lenses. This ensures that the synthesized image exhibits geometric shapes and color characteristics consistent with real optical laws, thereby maximizing the reproduction of the image quality of a real train.

[0106] Obviously, the technical solution provided in this embodiment can make the fault-added image present distortion and edge color difference that are highly consistent with the real vehicle wide-angle lens.

[0107] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above step: acquiring the target image includes: Acquire raw image sets of the pantograph under different operating environments; the different operating environments of the pantograph include: tunnel operating environment, maintenance depot operating environment and open-air operating environment; The original image set of the pantograph under the target operating environment is selected, and one image is selected from the original image set of the pantograph under the target operating environment to obtain the target selected image; the target operating environment is any operating environment of the pantograph under different operating environments.

[0108] In practical applications, the operating environments of pantographs are complex and diverse, with significant differences in lighting conditions and background features under different environments. In this embodiment, a set of original images of the pantograph under different operating environments is pre-collected, including tunnel operating environments, maintenance depot operating environments, and open-air operating environments. Then, the original image set of the pantograph under the target operating environment is filtered according to actual needs, and one image is selected from it to obtain the target selected image. Thus, anomaly samples of the pantograph under the target operating environment can be generated based on the target selected image.

[0109] Obviously, the technical solution provided in this embodiment can enable the generated abnormal samples to cover various operating environments of the pantograph. This not only improves the diversity of abnormal samples, but also enhances the generalization ability and robustness of the detection model under complex and severe weather conditions.

[0110] In a preferred embodiment, the above method further includes: The background image of the target screening image is determined by using histogram matching parameters and noise distribution model, and the target screening background is determined to be the background image of the pantograph in the target operating environment.

[0111] In practical applications, batch automatic synthesis of pantograph structural fault samples can also be achieved based on multi-scene background streams. Specifically, after obtaining the target screening image, histogram matching parameters and noise distribution models can be used to determine the background image of the target screening image, thus obtaining the target screening background. Then, the target screening background can be determined as the background image of the pantograph under the target operating environment. Finally, the same fault foreground image can be fused with the background images of the pantograph under different operating environments to generate fault-added images of the pantograph in batches under various operating environments.

[0112] Obviously, the technical solution provided in this embodiment can more quickly transfer the same fault mode of the pantograph to different operating backgrounds of the pantograph, thereby achieving an exponential increase in the scale of abnormal sample data, which can significantly improve the adaptability of the detection model to complex environments.

[0113] In practical applications, in order to generate abnormal samples of pantographs more conveniently and quickly, an abnormal sample generation system corresponding to the abnormal sample generation method provided in the aforementioned embodiment can be built, and the various modules in the abnormal sample generation system can be used to implement the various steps in the above-mentioned abnormal sample generation method in a streamlined manner.

[0114] Please see Figure 2 , Figure 2 This is a schematic flowchart of an abnormal sample generation system provided in an embodiment of the present invention. Figure 2 In the flowchart shown, when the user inputs a target image (any image in the original pantograph image set), the anomaly sample generation system will select and determine the type of anomaly sample the user wants to generate based on the user's configuration instructions for the anomaly sample generation task through the task configuration and routing module.

[0115] When the anomaly sample generation system determines that the user intends to add a foreign object to the target image, the region extraction module within the system extracts the target region for which the foreign object will be added based on the user's command and dynamically crops this region. To ensure the target region conforms to the preset standard format of the multimodal image editing model, the region extraction module also fills in the white borders of the target region and scales it using a high-fidelity scaling method to obtain the scaled target region. Next, the foreign object addition module uses the attention cross-referencing algorithm in Transformer to align the generated text with the scaled target region and optimizes the algorithm using SVD. After processing by the foreign object addition module, the image with the foreign object added is obtained.

[0116] When the anomaly sample generation system determines that the user wants to add a pantograph fault to the target image, the fault generation module builds a digital twin model corresponding to the pantograph based on a kinematic simulation algorithm with skeletal binding. It then uses Blender to modify the digital twin model to simulate the pantograph fault, thus obtaining a fault foreground image. Next, the image fusion module extracts the background image from the target image to obtain the target background image. Simultaneously, the image fusion module stores the extracted background image in a background library for use in subsequent processes. Finally, the image fusion module performs consistency correction on the fault foreground image and the target background image, and then fuses the fault foreground image with the target background image to obtain the fault-added image.

[0117] Once the images of foreign objects and faults are obtained, it is equivalent to acquiring abnormal samples of the pantograph. After acquiring these abnormal samples, the detection model in the intelligent inspection system can be trained using these samples, thereby improving the detection accuracy of the detection model in terms of the pantograph's operating status.

[0118] Please see Figure 3 , Figure 3 This is a flowchart illustrating the process of adding an image to generate a foreign object image. First, the target area for adding the foreign object needs to be extracted from the target image according to user instructions, and then cropped according to a preset standard format. If the target area is larger than the preset standard format, it needs to be downsampled based on the size ratio of the target image to the target area to reduce its size. If the target area is smaller than the preset standard format, it needs to be upsampled based on the size ratio of the target image to the target area to enlarge its size. After cropping and scaling the target area to obtain the scaled target area, the white borders in the scaled target area also need to be filled.

[0119] Next, the attention cross-referencing algorithm in Transformer can be used to align the target scaling region with the foreign object generation sample, and SVD can be used to optimize the attention cross-referencing algorithm. During this process, the visual encoder extracts image features from the target scaling region and participates in the attention cross-referencing algorithm's processing. Using the attention cross-referencing algorithm to align the target scaling region with the foreign object generation sample generates a high-fidelity foreign object addition region. Finally, the foreign object addition region and the target image are fused to obtain the foreign object addition image.

[0120] Please see Figure 4 , Figure 4This is a flowchart illustrating the process of generating fault-adding images. When generating fault-adding images, the digital twin model of the pantograph is first loaded into Blender. Based on the fault type of the pantograph, the digital twin model is used to simulate the fault occurring in the pantograph. When simulating the fault using the digital twin model, kinematic simulation algorithms based on skeletal binding can be used to modify the digital twin model of the pantograph to simulate the fault, such as breakage, wear, deformation, or bending of the pantograph.

[0121] When a pantograph malfunction is simulated and a malfunction foreground image is obtained, the PBR material system is used to optimize the malfunction foreground image and perform optical distortion correction. This allows for the rendering of a more realistic malfunction foreground image that better matches the appearance of a real wide-angle vehicle camera lens. Next, based on the noise distribution of the target background image, Gaussian noise is added to the malfunction foreground image to eliminate compositing artifacts. Finally, the Alpha blending node in the compositor is used to merge the target background image and the malfunction foreground image together, resulting in the malfunction-added image.

[0122] The foregoing embodiments described in detail a method for generating abnormal samples. This invention also provides an embodiment of an apparatus for generating abnormal samples. It should be noted that this invention describes the apparatus embodiment from the perspective of functional modules.

[0123] Please see Figure 5 , Figure 5 This is a structural diagram of an anomalous sample generation device provided in an embodiment of the present invention. The device includes: Image acquisition module 21 is used to acquire a target image; the target image is any image in the original image set of the pantograph; The region extraction module 22 is used to extract the target region of the target image to which the foreign object is to be added, and to scale the target region according to a preset standard format to obtain the target scaled region; The foreign object addition module 23 is used to align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and to fuse the foreign object addition area with the target image to obtain the foreign object addition image; The fault generation module 24 is used to build a digital twin model corresponding to the pantograph based on the kinematic simulation algorithm of skeleton binding, and to modify the digital twin model using Blender to simulate the fault of the pantograph and obtain a fault foreground image. The image fusion module 25 is used to fuse the fault foreground image with the target background image to obtain a fault-added image; the target background image is the background image in the target image; The sample determination module 26 is used to determine both the foreign object added image and the fault added image as abnormal samples of the pantograph.

[0124] Preferably, the region extraction module 22 includes: The region annotation submodule is used to annotate the target region in the target image where the foreign object is to be added; The region extraction submodule is used to extract the target region based on the proportional relationship between the target region and the target image, and using an adaptive interpolation algorithm.

[0125] Preferably, the region labeling submodule includes: The region labeling unit is used to label the target region in the target image to which the foreign object is to be added, according to user instructions.

[0126] Preferably, the region extraction module 22 includes: The region scaling submodule is used to scale the target region based on the Lanczos interpolation algorithm and according to the preset standard format to obtain the target scaled region.

[0127] Preferred options also include: The white border fill submodule is used to fill the white border area if there is a white border area in the target scaling area.

[0128] Preferably, the foreign object adding module 23 includes: The text alignment submodule is used to align the target scaling region with the text generated by the foreign object using the attention cross algorithm in Transformer, thereby obtaining the foreign object addition region.

[0129] Preferred options also include: The algorithm optimization submodule is used to optimize the attention cross algorithm using SVD.

[0130] Preferably, the fault generation module 24 includes: The parameter modification submodule is used to modify the setting parameters in the digital twin model based on user instructions using the Blender to simulate the fault of the pantograph and obtain the fault foreground image.

[0131] Preferably, the fault generation module 24 includes: The device deformation submodule is used to perform bending deformation on a specified device in the digital twin model based on user instructions using the Blender to simulate the fault that occurs in the pantograph and obtain the fault foreground image.

[0132] Preferred options also include: The material optimization submodule is used to optimize the faulty foreground image using the PBR material system.

[0133] Preferably, the image fusion module 25 includes: The image fusion submodule is used to fuse the fault foreground image and the target background image using the Alpha blending node in the synthesizer to obtain the fault-added image; the synthesizer is a component of the Blender.

[0134] Preferred options also include: The noise addition submodule is used to add Gaussian noise to the fault foreground image according to the noise distribution pattern of the target background image.

[0135] Preferred options also include: The background extraction submodule is used to remove non-environmental interference features from the target image to obtain a first processed image; extract the support structure of the pantograph in the first processed image and remove the support structure from the first processed image to obtain a second processed image; remove the base and connecting rod of the pantograph in the second processed image to obtain the target background image.

[0136] Preferred options also include: The residual scanning submodule is used to perform a global residual scan on the target background image to remove artifacts in the target background image.

[0137] Preferred options also include: The distortion correction submodule is used to perform optical distortion correction on the image added due to the fault.

[0138] Preferably, the image acquisition module 21 includes: The image set acquisition submodule is used to acquire the original image set of the pantograph under different operating environments; the different operating environments of the pantograph include: tunnel operating environment, maintenance depot operating environment and open-air operating environment; The image filtering submodule is used to filter the original image set of the pantograph under the target operating environment, and to filter one image from the original image set of the pantograph under the target operating environment to obtain the target filtered image; the target operating environment is any operating environment of the pantograph under different operating environments.

[0139] Preferred options also include: The background determination submodule is used to determine the background image of the target screening image using histogram matching parameters and noise distribution model, obtain the target screening background, and determine the target screening background as the background image of the pantograph in the target operating environment.

[0140] The abnormal sample generation apparatus provided in this embodiment of the invention has the beneficial effects of the abnormal sample generation method disclosed above.

[0141] Please see Figure 6 , Figure 6 This is a structural diagram of an abnormal sample generation device provided in an embodiment of the present invention. The device includes: Memory 31 is used to store computer programs; The processor 32 is configured to implement the steps of a method for generating an anomaly sample as disclosed above when executing the computer program.

[0142] The processor 32 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 32 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 32 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 32 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen and for performing parallel computing tasks required for multimodal image editing of large models and 3D rendering. In some embodiments, the processor 32 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0143] The memory 31 may include one or more computer-readable storage media, which may be non-transitory. The memory 31 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 31 is used to store at least the following computer program 301, which, after being loaded and executed by the processor 32, is capable of implementing the relevant steps of the abnormal sample generation method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 31 may also include an operating system 302 and data 303, and the storage method may be temporary or permanent storage. The operating system 302 may include Windows, Unix, Linux, etc. The data 303 may include, but is not limited to, the data involved in the abnormal sample generation method.

[0144] In some embodiments, the device for generating abnormal samples may further include a display screen 33, an input / output interface 34, a communication interface 35, a power supply 36, and a communication bus 37.

[0145] The abnormal sample generation device provided in this embodiment of the invention has the beneficial effects of the abnormal sample generation method disclosed above.

[0146] Accordingly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an abnormal sample generation method as disclosed above.

[0147] The computer-readable storage medium provided in this embodiment of the invention has the beneficial effects of the aforementioned method for generating abnormal samples.

[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0149] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The above provides a detailed description of the method, apparatus, device, and medium for generating abnormal samples provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating abnormal samples, characterized in that, include: Acquire the target image; The target image is any one image from the original image set of the pantograph; Extract the target region of the target image from which the foreign object is to be added, and scale the target region according to a preset standard format to obtain the target scaled region; Align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and then fuse the foreign object addition area with the target image to obtain the foreign object addition image; A kinematic simulation algorithm based on skeletal binding is used to build a digital twin model corresponding to the pantograph, and Blender is used to modify the digital twin model to simulate the fault of the pantograph and obtain a fault foreground image. The fault foreground image is fused with the target background image to obtain the fault-added image; The target background image is the background image in the target image; Both the foreign object image and the fault image are identified as abnormal samples of the pantograph.

2. The method for generating abnormal samples according to claim 1, characterized in that, The step of extracting the target region in the target image from which the foreign object is to be added includes: Mark the target area in the target image where the foreign object is to be added; Based on the proportional relationship between the target region and the target image, the target region is extracted using an adaptive interpolation algorithm.

3. The method for generating abnormal samples according to claim 2, characterized in that, The process of marking the target region in the target image to which the foreign object is to be added includes: The target area in the target image to which the foreign object is to be added is marked according to the user's instructions.

4. The method for generating abnormal samples according to claim 1, characterized in that, The process of scaling the target region according to a preset standard format to obtain a scaled target region includes: The target scaled region is obtained by scaling the target region according to the preset standard format based on the Lanczos interpolation algorithm.

5. The method for generating abnormal samples according to claim 4, characterized in that, Also includes: If there is a white border area in the target scaling area, then the white border area is filled.

6. The method for generating abnormal samples according to claim 1, characterized in that, The step of aligning the target scaling area with the foreign object generation text to obtain the foreign object addition area includes: The target scaling region is aligned with the foreign object generated text using the attention cross-interaction algorithm in Transformer to obtain the foreign object addition region.

7. The method for generating abnormal samples according to claim 6, characterized in that, Also includes: The attention crossover algorithm is optimized using SVD.

8. The method for generating abnormal samples according to claim 1, characterized in that, The modification of the digital twin model using Blender to simulate a fault in the pantograph and obtain a fault foreground image includes: Based on user instructions, the settings parameters in the digital twin model are modified using the Blender to simulate the fault that occurs in the pantograph, and the fault foreground image is obtained.

9. The method for generating abnormal samples according to claim 1, characterized in that, The modification of the digital twin model using Blender to simulate a fault in the pantograph and obtain a fault foreground image includes: Based on user instructions, the specified device in the digital twin model is bent and deformed using the Blender to simulate the fault of the pantograph and obtain the fault foreground image.

10. The method for generating abnormal samples according to claim 1, characterized in that, Also includes: The fault foreground image was optimized using the PBR material system.

11. The method for generating abnormal samples according to claim 1, characterized in that, The step of fusing the fault foreground image with the target background image to obtain the fault-added image includes: The fault foreground image and the target background image are fused using the Alpha blending node in the synthesizer to obtain the fault-added image; the synthesizer is a component of the Blender.

12. The method for generating abnormal samples according to claim 11, characterized in that, Also includes: Gaussian noise is added to the fault foreground image based on the noise distribution pattern of the target background image.

13. The method for generating abnormal samples according to claim 1, characterized in that, Also includes: Remove non-environmental interference features from the target image to obtain the first processed image; Extract the pantograph support structure from the first processed image and remove the support structure from the first processed image to obtain the second processed image; The pantograph base and connecting rod are removed from the second processed image to obtain the target background image.

14. The method for generating abnormal samples according to claim 13, characterized in that, Also includes: A global residual scan is performed on the target background image to remove artifacts from the target background image.

15. The method for generating abnormal samples according to claim 14, characterized in that, Also includes: Optical distortion correction is performed on the image added to the faulty image.

16. The method for generating abnormal samples according to claim 1, characterized in that, The acquisition of the target image includes: Acquire original image sets of the pantograph under different operating environments; the different operating environments of the pantograph include: tunnel operating environment, maintenance depot operating environment and open-air operating environment; The original image set of the pantograph under the target operating environment is filtered, and one image is selected from the original image set of the pantograph under the target operating environment to obtain the target selected image; the target operating environment is any operating environment of the pantograph under different operating environments.

17. The method for generating abnormal samples according to claim 16, characterized in that, Also includes: The background image of the target screening image is determined by using histogram matching parameters and noise distribution model, thus obtaining the target screening background, and the target screening background is determined as the background image of the pantograph in the target operating environment.

18. An apparatus for generating abnormal samples, characterized in that, include: The image acquisition module is used to acquire the target image; The target image is any one image from the original image set of the pantograph; The region extraction module is used to extract the target region of the target image to which the foreign object is to be added, and to scale the target region according to a preset standard format to obtain the target scaled region; The foreign object addition module is used to align the target scaling area with the foreign object generation text to obtain the foreign object addition area, and to fuse the foreign object addition area with the target image to obtain the foreign object addition image; The fault generation module is used to build a digital twin model corresponding to the pantograph based on the kinematic simulation algorithm of skeleton binding, and to modify the digital twin model using Blender to simulate the faults that occur in the pantograph and obtain a fault foreground image. An image fusion module is used to fuse the fault foreground image with the target background image to obtain a fault-added image; The target background image is the background image in the target image; The sample determination module is used to determine both the foreign object added image and the fault added image as abnormal samples of the pantograph.

19. An anomalous sample generation device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a method for generating an anomalous sample as described in any one of claims 1 to 17 when executing the computer program.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a method for generating an anomalous sample as described in any one of claims 1 to 17.

Images