A method, apparatus and electronic device for detecting defects on the surface of a cavity.

Abstract

This application relates to the field of image analysis technology, and provides a method, apparatus, and electronic device for detecting defects on the surface of a cavity, including: acquiring a target image to be detected and corresponding text prompt information to construct an image-text modal input pair; extracting image feature vectors and text feature vectors from the target image; and introducing HOPLORA pairs. n img and n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vector. n img’ and text feature vectors n txt’ Calculate the image-text similarity score s corresponding to the target image and generate an anomaly scoring map. A To identify defect areas, a visual language model is used to enhance the model's understanding of the valve body structure semantics through image-text modal fusion; the HOPLoRA module performs efficient low-rank parameter adaptation, improving cross-scene detection performance; and pixel-level anomaly scoring maps are output based on image-text similarity to achieve fine-grained defect localization. This method significantly improves the accuracy of defect detection on the valve body's internal cavity surface, adapts to complex working conditions, and enhances the model's scalability.

Description

Technical Field

[0001] This invention belongs to the field of image analysis technology, and in particular relates to a method, device and electronic equipment for detecting defects on the surface of a cavity. Background Technology

[0002] With the rapid development of intelligent manufacturing and quality control technologies, cavities, as crucial components in industrial equipment used to control fluid flow, direction, and pressure, are increasingly attracting attention regarding the surface integrity and defect detection of their internal structures. Due to factors such as the geometric complexity, uneven illumination, and variations in surface materials within cavities, traditional defect detection methods based on manual visual inspection or rule-based image algorithms struggle to accurately identify potential defects. This is particularly true for diverse defect types such as microcracks, corrosion spots, and foreign matter residues, where detection accuracy is low and the false negative rate is high.

[0003] With the continuous development of computer technology, deep learning-based image analysis methods have attracted increasing attention. Among them, convolutional neural networks and transformer models can achieve automatic extraction and classification of image features. However, due to the scarcity of cavity defect samples in industrial fields and the high cost of annotation, problems such as domain bias and semantic expression diversity exist in actual detection. This leads to poor adaptability and weak generalization ability of models in cross-scene applications. Furthermore, existing commonly used methods often lack understanding and fusion of cavity state semantics, which limits the ability of detection algorithms to identify defects in complex backgrounds. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus and electronic device for detecting defects on the surface of a cavity, in order to solve the problems of poor adaptability, weak generalization ability and limited defect recognition ability of existing defect detection models or algorithms in cross-scenario applications.

[0005] A first aspect of the present invention provides a method for detecting defects on the surface of a cavity, the method comprising:

[0006] The target image to be detected and the corresponding text prompt information are obtained, and a graphic-text modal input pair is constructed through a visual language model.

[0007] Based on the image-text modal input pair, the image feature vector of the target image is extracted by a visual encoder and a text encoder. n img With text feature vectors n txt ;

[0008] The image feature vector is processed using the HOPLoRA multi-expert low-rank adaptation module. n img With the text feature vector ntxt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors ;

[0009] Based on the transformed image feature vector and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ;

[0010] Through the anomaly scoring graph A Defects present in the target image are identified.

[0011] A second aspect of the present invention provides a cavity surface defect detection device, the cavity surface defect detection device comprising:

[0012] The image acquisition module is used to acquire the target image to be detected and the text prompt information corresponding to the target image, and to construct a graphic-text modal input pair through a visual language model;

[0013] The feature vector conversion module is used to extract image feature vectors of the target image based on the image-text modal input pair through a visual encoder and a text encoder. n img With text feature vectors n txt The image feature vector is processed using the HOPLoRA multi-expert low-rank adaptation module. n img With the text feature vector n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors ;

[0014] The rating image generation module is used to generate a rating image based on the transformed image feature vector. and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ;

[0015] The defect judgment module is used to determine the defect score based on the anomaly rating graph. A Defects present in the target image are identified.

[0016] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any of the first aspects above.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects above.

[0018] A fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to perform the steps of the method described in any of the first aspects.

[0019] The beneficial effects of this invention compared to the prior art are as follows: This application acquires the target image to be detected and the corresponding text prompt information, and constructs a graphic-text modal input pair through a visual language model; based on the graphic-text modal input pair, it extracts the image feature vector of the target image through a visual encoder and a text encoder. n img With text feature vectors n txt Then, the HOPLoRA multi-expert low-rank adaptation module is introduced to adapt the image feature vectors. n img With text feature vectors n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. n img’ and text feature vectors n txt’ Furthermore, based on the transformed image feature vector n img’ and text feature vectors n txt’ Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ; and through anomaly scoring charts AThe method identifies defective regions within the target image. This process utilizes a visual language model to achieve image-text modal fusion, enhancing the model's understanding of the valve body's structural semantics. The HOPLoRA module is introduced for efficient low-rank parameter adaptation, improving cross-scene detection performance and model lightweightness. Pixel-level anomaly scoring maps are output based on image-text similarity, enabling fine-grained defect localization. The method supports efficient deployment under unsupervised or minimally labeled conditions, making it suitable for detection tasks where defect samples are scarce in real-world industrial production environments. This approach significantly improves the accuracy of valve body cavity surface defect detection, adapts to complex operating conditions, and enhances model scalability.

[0020] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating the implementation process of the cavity surface defect detection method provided in this embodiment of the invention;

[0023] Figure 2 This is a schematic diagram of a HOPLoRA module provided in another embodiment of this application;

[0024] Figure 3 This is a diagram showing the results of valve body cavity surface defect detection based on HOPLoRA;

[0025] Figure 4 This is a schematic diagram of a cavity surface defect detection device provided in another embodiment of this application;

[0026] Figure 5 This is a schematic diagram of a terminal device provided in another embodiment of the present invention. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0028] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0029] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0031] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0033] The beneficial effects of this invention compared to the prior art are as follows: This application acquires the target image to be detected and the corresponding text prompt information, and constructs a graphic-text modal input pair through a visual language model; based on the graphic-text modal input pair, it extracts the image feature vector of the target image through a visual encoder and a text encoder. n img With text feature vectors n txt Then, the HOPLoRA multi-expert low-rank adaptation module is introduced to adapt the image feature vectors. n img With text feature vectors n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors Furthermore, based on the transformed image feature vector and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ; and through anomaly scoring charts A The method identifies defective regions within the target image. This process utilizes a visual language model to achieve image-text modal fusion, enhancing the model's understanding of the valve body's structural semantics. The HOPLoRA module is introduced for efficient low-rank parameter adaptation, improving cross-scene detection performance and model lightweightness. Pixel-level anomaly scoring maps are output based on image-text similarity, enabling fine-grained defect localization. The method supports efficient deployment under unsupervised or minimally labeled conditions, making it suitable for detection tasks where defect samples are scarce in real-world industrial production environments. This approach significantly improves the accuracy of valve body cavity surface defect detection, adapts to complex operating conditions, and enhances model scalability.

[0034] Please refer to details. Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation process of a cavity surface defect detection method according to an embodiment of this application. The cavity surface defect detection method may include:

[0035] Step 101: Obtain the target image to be detected and the text prompt information corresponding to the target image, and construct a graphic-text modal input pair through a visual language model.

[0036] The embodiments provided in this application acquire a high-resolution (e.g., 1920*1080) image for characterizing the state of the valve body cavity, which can also be referred to as the target image to be detected, and the corresponding text prompt information for this image. For example, the text prompt information can be represented as: an image with pits; a normal image, etc. Then, a graphic-text modal input pair is constructed through a visual language model, specifically including:

[0037] Image modalities and text modalities may have different dimensions, therefore it is necessary to implement the image modalities (i.e., the target image) based on visual language models (such as VLMo, Flamingo, BEIT3, etc.). With text modality (i.e., text prompts) Domain alignment.

[0038] The text prompt Using a template-based constructor, the form is as follows:

[0039] ;

[0040] Step 102: Based on the image-text modal input pair, extract the image feature vector of the target image using a visual encoder and a text encoder. n img With text feature vectors n txt ;

[0041] Modal input pair (Target image) and (Text prompts are information), which are processed by the visual encoder. With text encoder Extract the feature vectors separately.

[0042] ;

[0043] Step 103: The image feature vector is processed by the HOPLORA multi-expert low-rank adaptation module. n img With the text feature vector n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vector and text feature vector, respectively;

[0044] The embodiments provided in this application introduce the HOPLoRA module to perform domain-adaptive transformation of modal features:

[0045]

[0046] The specific processing procedure of the HOPLoRA module is "feature preprocessing → orthogonal projection → gated fusion calculation" structure, such as... Figure 2 As shown. Its input is the feature vector of the image and text encoder. The output is the features after neighborhood alignment. .

[0047] ① Feature preprocessing. Layer normalization is performed on the input image feature vector and text feature vector to obtain... Then sent in A sub-network of experts .

[0048] ② Orthogonal projection. The expert subnetwork uses low-rank coding to calculate the increment. :

[0049] ;

[0050] in, , , , For low-rank descending matrices, For low-rank uprow matrices, This is a nonlinear activation. To suppress collinearity and improve separability, [the following is used]: Perform orthographic projection processing: ;

[0051] in, It is an orthogonal basis.

[0052] ③ Gating weight calculation. A Top-k sparse gating system is used, retaining only the system with the largest weight. One expert participated in the fusion, and the rest were set to zero. (Input...) To query, calculate the gating weights for each expert. : ;

[0053] in, For learnable parameters, This is the temperature coefficient.

[0054] ④ Feature vector calculation. Image feature vectors n img With the text feature vector n txt Respectively with gating weights Performing the dot product operation yields the transformed image feature vector. and text feature vectors .

[0055] Step 104: Calculate the image-text similarity score corresponding to the target image based on the transformed image feature vector and text feature vector. Generate anomaly scoring chart A .

[0056] Optionally, the abnormality scoring graph is used. A Determining defects present in the target image includes:

[0057] The abnormal threshold θ1 is preset according to the defect detection accuracy;

[0058] The anomaly scoring graph The location where the value is greater than the threshold θ1 is determined to be a defect region;

[0059] The anomaly scoring graph Positions within the range that are smaller than the threshold θ1 are determined to be defect-free regions.

[0060] Step 105, using the anomaly scoring graph A Defects present in the target image are identified.

[0061] In the embodiments provided in this application, the image-text fusion similarity calculation can adopt the dot product attention mechanism, as follows: ;

[0062] in

[0063] Anomaly scoring chart The similarity set of all patches Generated through fusion:

[0064] ;

[0065] If abnormal scoring chart Greater than the pre-screening threshold θ1, i.e. If the value is greater than θ1, then an anomaly determination is output; otherwise... When <θ1, output normal judgment.

[0066] Optionally, determining the defects present in the target image through the anomaly scoring map includes:

[0067] The anomaly scoring map is converted into a heatmap and a mask map;

[0068] Defects in the target image are detected and located using the heatmap and mask image.

[0069] like Figure 3 As shown, Figure 3 Image A in the image is the acquired image to be detected. Figure 3 The defective areas are marked with a blue box in Figure B. Figure 3 The red area in Figure C corresponds to the blue box area in Figure B, which is the defect area detected by the heat map and mask map.

[0070] This application acquires the target image to be detected and the corresponding text prompt information, constructs a graphic-text modal input pair using a visual language model, and extracts the image feature vector of the target image based on the graphic-text modal input pair using a visual encoder and a text encoder. n img With text feature vectors n txt Then, the HOPLoRA multi-expert low-rank adaptation module is introduced to adapt the image feature vectors. n img With text feature vectors n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors Furthermore, based on the transformed image feature vector and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ; and through anomaly scoring chartsA The method identifies defective regions within the target image. This process utilizes a visual language model to achieve image-text modal fusion, enhancing the model's understanding of the valve body's structural semantics. The HOPLoRA module is introduced for efficient low-rank parameter adaptation, improving cross-scene detection performance and model lightweightness. Pixel-level anomaly scoring maps are output based on image-text similarity, enabling fine-grained defect localization. The method supports efficient deployment under unsupervised or minimally labeled conditions, making it suitable for detection tasks where defect samples are scarce in real-world industrial production environments. This approach significantly improves the accuracy of valve body cavity surface defect detection, adapts to complex operating conditions, and enhances model scalability.

[0071] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0072] Another embodiment of this application also provides a device for detecting defects on the surface of a cavity, such as... Figure 4 As shown, the cavity surface defect detection device 4 provided in this embodiment includes:

[0073] Image acquisition module 41 is used to acquire the target image to be detected and the text prompt information corresponding to the target image, and to construct a graphic-text modal input pair through a visual language model;

[0074] Feature vector conversion module 42 is used to extract image feature vectors of the target image based on the image-text modal input pair through a visual encoder and a text encoder. n img With text feature vectors n txt The image feature vector is processed using the HOPLoRA multi-expert low-rank adaptation module. n img With the text feature vector n txt Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors ;

[0075] The scoring image generation module 43 is used to generate a scoring image based on the transformed image feature vector. and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart A ;

[0076] Defect judgment module 44 is used to determine the defect through the anomaly scoring graph. A Defects present in the target image are identified.

[0077] The specific implementation process of each module in the above-mentioned cavity surface defect detection device 4 can be referred to the flow of the cavity surface defect detection method, and will not be repeated here.

[0078] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Figure 5 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 (Only one is shown in the diagram) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, which, when executing the computer program 52, implements the steps in any of the above-described music collection method embodiments.

[0079] The electronic device 5 can be a desktop computer, laptop, PDA, or other Bluetooth device with audio playback capabilities. This electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0080] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0081] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may be an external storage device of the electronic device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 5. In one embodiment, the memory 51 may include both internal storage units and external storage devices of the electronic device 5. The memory 51 is used to store operating systems, applications, boot loaders, data, and other programs, such as the program code of the computer program. The memory 51 may also be used to temporarily store data that has been output or will be output.

[0082] For example, the electronic device may be a station (ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a set-top box (STB), customer premises equipment (CPE), and / or other devices for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved Public Land Mobile Network (PLMN) networks.

[0083] Communication modules can provide solutions for communication applications on network devices, including Wireless Local Area Networks (WLANs) (such as Wi-Fi networks), Bluetooth, Zigbee, mobile communication networks, Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. A communication module can be one or more devices integrating at least one communication processing module. The communication module may include an antenna, which can have a single element or be an antenna array with multiple elements. The communication module can receive electromagnetic waves through the antenna, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor. The communication module can also receive signals to be transmitted from the processor, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0084] The power management module can receive input from the battery and / or charger to power the processor, memory, and communication modules.

[0085] It should be noted that the above Figure 5 This does not constitute a limitation on the structure of electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, electronic device 5 may also include a display screen, indicator lights, motors, controls (such as buttons), gyroscope sensors, accelerometer sensors, etc.

[0086] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0088] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0089] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0090] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0091] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0092] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0094] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting defects on the surface of a cavity, characterized in that, The detection method includes: The process involves acquiring the target image to be detected and its corresponding text prompt information, constructing a text-image modal input pair using a visual language model, and extracting the image feature vector of the target image based on the text-image modal input pair using a visual encoder and a text encoder. With text feature vectors The image feature vector is processed using the HOPLoRA multi-expert low-rank adaptation module. With the text feature vector Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors ,in: Based on the transformed image feature vector and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart ; through the aforementioned anomaly scoring chart Identify the defects present in the target image; The image feature vector is processed by the HOPLoRA multi-expert low-rank adaptation module. With the text feature vector Performing domain transformation and semantic alignment includes: normalizing the image feature vector and the text feature vector to obtain... Then sent in A sub-network of experts Orthogonal projection; the expert subnetwork uses low-rank coding to calculate the increment. ; ;in: For low-rank mapping downlink matrix, For low-rank mapping up row matrix, To suppress collinearity and improve separability by nonlinear activation, Perform orthographic projection processing: ;in, For low-rank transformation matrices, An orthogonal basis is established; the gating weights of each expert network are calculated using gating weights. ; to extract image feature vectors and the text feature vector Respectively with gating weights Perform dot product operations, perform domain transformation, and semantic alignment.

2. The method for detecting defects on the surface of a cavity as described in claim 1, characterized in that, The gating weights of each expert network are calculated using gating weights. This includes: using Top-k sparse gating, retaining only the one with the largest weight. One expert participated in the integration, and the rest were set to zero; input... To query, calculate the gating weights for each expert. ; ;in, For learnable parameters, This is the temperature coefficient.

3. The method for detecting defects on the surface of a cavity as described in claim 1, characterized in that, The abnormal scoring graph Determining defects present in the target image includes: The abnormal threshold θ1 is preset according to the defect detection accuracy; The anomaly scoring graph The location where the value is greater than the threshold θ1 is determined to be a defect region; The anomaly scoring graph The positions that are smaller than the threshold θ1 are determined to be defect-free regions.

4. The method for detecting defects on the surface of a cavity as described in claim 1, characterized in that, The step of determining the defects present in the target image through the anomaly scoring map includes: The anomaly score map is converted into a heatmap and a mask map; the defects in the target image are detected and located using the heatmap and mask map.

5. A device for detecting defects on the surface of a cavity, characterized in that, The cavity surface defect detection device includes: The image acquisition module is used to acquire the target image to be detected and the corresponding text prompt information, and constructs an image-text modal input pair through a visual language model; the feature vector conversion module is used to extract the image feature vector of the target image based on the image-text modal input pair through a visual encoder and a text encoder. With text feature vectors The image feature vector is processed using the HOPLoRA multi-expert low-rank adaptation module. With the text feature vector Perform domain transformation and semantic alignment to obtain the transformed image feature vectors. and text feature vectors ,in: The scoring image generation module is used to generate a scoring image based on the transformed image feature vector. and text feature vectors Calculate the image-text similarity score corresponding to the target image. Generate anomaly scoring chart ; The defect judgment module is used to determine the defect score based on the anomaly rating graph. Determine the defects present in the target image; the image feature vector is then processed using the HOPLORA multi-expert low-rank adaptation module. With the text feature vector The row domain transformation and semantic alignment include: normalizing the image feature vector and the text feature vector to obtain... Then sent in A sub-network of experts Orthogonal projection; the expert subnetwork uses low-rank coding to calculate the increment. : ; in, For low-rank mapping downlink matrix, For low-rank mapping up row matrix, To suppress collinearity and improve separability by nonlinear activation, Perform orthographic projection processing: ;in, For low-rank transformation matrices, An orthogonal basis is established; the gating weights of each expert network are calculated using gating weights. ; to extract image feature vectors and the text feature vector Respectively with gating weights Perform dot product operations, perform domain transformation, and semantic alignment.

6. The cavity surface defect detection device as described in claim 5, characterized in that, The defect detection module is specifically used for: The abnormal threshold θ1 is preset according to the defect detection accuracy; The anomaly scoring graph The location where the value is greater than the threshold θ1 is determined to be a defect region; The anomaly scoring graph The positions that are smaller than the threshold θ1 are determined to be defect-free regions.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images