Cross-flow fan quality detection method and device, storage medium and computer program product

Abstract

The present disclosure relates to a cross-flow fan quality detection method and device, a storage medium and a computer program product. The method comprises: in response to a detection trigger signal of a cross-flow fan placed on a rotating platform, collecting multi-view images of the cross-flow fan; in response to completion of single-segment multi-view image collection, controlling a sliding table to move a camera module to corresponding multiple shooting positions according to a preset step distance, repeatedly collecting images to obtain multi-segment multi-view images; in response to completion of multi-segment multi-view image collection, performing regional cropping on the multi-segment multi-view images, and outputting a fan blade region subgraph and a center plate region subgraph; inputting the fan blade region subgraph and the center plate region subgraph into a corresponding angle defect detection model respectively, and outputting abnormal region detection results of multiple subgraphs; and drawing a visual detection result of the cross-flow fan according to the abnormal region detection results of the multiple subgraphs, thereby improving detection accuracy and reducing the calculation complexity of model reasoning.

Description

Technical Field

[0001] This disclosure relates to the field of quality inspection technology, and in particular to a method, apparatus, storage medium, and computer program product for quality inspection of cross-flow fans. Background Technology

[0002] Cross-flow fans are a core component of air conditioners. After production, the appearance quality of the manufactured parts needs to be inspected. In related technologies, the appearance quality inspection of cross-flow fans mainly uses traditional supervised deep learning solutions and industrial vision large model few sample solutions. However, the solutions of these technologies have insufficient adaptability, which leads to a decrease in the accuracy of appearance quality inspection of the parts. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, this disclosure provides a method, apparatus, storage medium and computer program product for testing the quality of cross-flow fans.

[0004] According to a first aspect of the present disclosure, a method for detecting the quality of a cross-flow fan is provided, comprising:

[0005] In response to a detection trigger signal that the cross-flow fan is placed on a rotating platform, a multi-view image of the cross-flow fan is acquired, wherein the multi-view image includes images acquired by a frontal view camera and a dual oblique view camera, and the multi-view image covers the circumferential surface of the fan blade tip and the split plate area on both sides of the fan.

[0006] In response to the completion of single-segment multi-view image acquisition, the control slide moves the camera module to multiple corresponding shooting positions according to the preset step distance, and repeatedly acquires images to obtain multiple segments of multi-view images.

[0007] In response to the completion of the acquisition of the multi-segment multi-view images, the multi-segment multi-view images are cropped to output a fan blade region sub-image and a mid-plate region sub-image.

[0008] The fan blade region sub-image and the center plate region sub-image are respectively input into the defect detection model of the corresponding viewpoint, and the abnormal region detection results of multiple sub-images are output.

[0009] The visualization results of the cross-flow fan are drawn based on the abnormal region detection results of the multiple sub-graphs.

[0010] In some embodiments, the control slide moves the camera module to multiple corresponding shooting positions according to a preset step distance, including: determining the corresponding slide movement step distance and number of segments based on the read cross-flow fan model information, and performing an image acquisition operation according to the slide movement step distance and number of segments; in response to the completion of each segment image acquisition, sending a movement command to the slide servo driver to control the slide to move the camera module to the corresponding shooting position; in response to the completion of multiple segment image acquisitions, sending a reset command to the slide servo driver to control the slide to move back to the initial position.

[0011] In some embodiments, cropping the multi-segment multi-view images to output fan blade region sub-images and median plate region sub-images includes: determining the position coordinates of the fan blade region and the median plate region in the multi-segment multi-view images; and overlapping and cropping the fan blade region and the median plate region based on the fan blade region position coordinates and the median plate region position coordinates according to a preset sub-image size, thereby outputting multiple fan blade region sub-images and median plate region sub-images.

[0012] In some embodiments, the method further includes: in response to a received model training instruction, loading unlabeled good sample images and weakly labeled defective sample images; performing data augmentation processing on the weakly labeled defective sample images to generate an augmented defective sample set; using the unlabeled good sample images and the augmented defective sample set as input, performing iterative model training in the meta-training phase, and outputting a meta-training base model; in response to a received fan adaptation instruction, loading the meta-training base model and unlabeled good samples of the fan to be adapted, performing parameter adjustment operations in the adaptation phase, and outputting an adapted defect detection model.

[0013] In some embodiments, the step of performing iterative training of the model during the meta-training phase to output a meta-training base model includes: determining multiple loss values ​​for each training iteration in response to each round of training iteration; obtaining a total loss value by weighted summation of the multiple loss values ​​according to preset weights; updating the model parameters by backpropagation based on the total loss value; and stopping model training and saving the updated target model parameters in response to the validation set loss value not decreasing for a consecutive preset number of rounds, and outputting the meta-training base model.

[0014] In some embodiments, the step of inputting the fan blade region sub-image and the mid-plate region sub-image into a defect detection model with corresponding viewpoints and outputting abnormal region detection results for multiple sub-images includes: inputting the fan blade region sub-image and the mid-plate region sub-image into a feature extraction network and outputting feature maps of multiple resolutions; performing weighted amplification processing on the features of the fan blade edge and the periphery of the mid-plate based on the feature maps of multiple resolutions; performing weighted fusion on the feature maps of multiple resolutions after feature amplification and outputting a fused feature map; outputting an abnormal region score map based on the fused feature map; and, in response to an abnormal region score exceeding a preset threshold, marking the abnormal region and outputting the mask coordinates and detection box coordinates of the abnormal region.

[0015] In some embodiments, after drawing the visualized detection results of the cross-flow fan based on the abnormal area detection results of the plurality of sub-graphs, the method further includes: in response to detecting defects in the cross-flow fan, identifying defect categories and statistically analyzing the percentage data of each type of defect; querying the defect process parameter mapping library based on the defect categories and the percentage data, and outputting corresponding process adjustment strategy information; and sending the process adjustment strategy information to the production line system and displaying it on the process monitoring interface.

[0016] In some embodiments, the step of drawing the visual detection result of the cross-flow fan based on the abnormal region detection results of the plurality of sub-images includes: in response to the same defect region being detected as abnormal in multiple viewpoints, determining the feature similarity of the defect regions in multiple viewpoints; in response to the feature similarity being higher than a preset threshold, merging the detection results of multiple viewpoints into the same defect record; and drawing the merged defect mask and detection box on the original image based on the defect record to generate the visual detection result.

[0017] According to a second aspect of the present disclosure, a cross-flow fan quality testing device is provided, comprising:

[0018] The acquisition module is used to acquire multi-view images of the cross-flow fan in response to the detection trigger signal of the cross-flow fan being placed on the rotating platform;

[0019] The control module is used to respond to the completion of single-segment multi-view image acquisition, and control the slide to move the camera module to multiple corresponding shooting positions according to the preset step distance, and repeatedly acquire images to obtain multiple segments of multi-view images.

[0020] The first output module is used to perform region cropping on the multi-segment multi-view image in response to the completion of the acquisition of the multi-segment multi-view image, and output the fan blade region sub-image and the center plate region sub-image.

[0021] The second output module is used to input the fan blade region sub-image and the middle plate region sub-image into the defect detection model of the corresponding viewpoint, and output the abnormal region detection results of multiple sub-images.

[0022] The drawing module is used to draw the visual detection results of the cross-flow fan based on the abnormal region detection results of the multiple sub-graphs.

[0023] According to a third aspect of the present disclosure, a cross-flow fan quality detection device is provided, comprising: a processor;

[0024] Memory used to store processor-executable instructions;

[0025] The processor is configured to execute the cross-flow fan quality detection method as described in the first aspect.

[0026] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by the computer, are used to implement the cross-flow fan quality detection method provided in the first aspect above.

[0027] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed, implements the cross-flow fan quality detection method provided in the first aspect.

[0028] This disclosure provides a method, apparatus, storage medium, and computer program product for quality inspection of a cross-flow fan. The method acquires multi-view images of the cross-flow fan in response to a placement trigger signal. After a single segment acquisition is completed, the slide is controlled to move to multiple shooting positions at a preset step distance for repeated acquisition, resulting in multiple segments of multi-view images. Region cropping is then performed to output sub-images of the fan blade area and the center plate area. These are then input into the corresponding defect detection model, and the abnormal area detection results are output and visualized. This solves the problem of missed defect detection in slender rotating structures like cross-flow fans due to blind spots in single-view imaging. By combining a frontal view and dual oblique view cameras, the circumferential surface of the fan blade segment and the center plate areas on both sides are fully covered. By controlling the slide to acquire multi-view images in segments according to a preset step distance, complete scanning of fans of different lengths is ensured. Region cropping decomposes large images into smaller images adapted to the model input, improving detection accuracy while reducing the computational complexity of model inference.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0031] Figure 1 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 1 .

[0032] Figure 2 This is a schematic diagram of camera layout according to some embodiments of the present disclosure.

[0033] Figure 3 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 2 .

[0034] Figure 4 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 3 .

[0035] Figure 5 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 4 .

[0036] Figure 6 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 5 .

[0037] Figure 7 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 6 .

[0038] Figure 8 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 7 .

[0039] Figure 9 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 8 .

[0040] Figure 10 This is a block diagram of a cross-flow fan quality testing device according to some embodiments of the present disclosure.

[0041] Figure 11 This is a physical block diagram illustrating a cross-flow fan quality testing device according to some embodiments of the present disclosure.

[0042] Figure 12 This is a block diagram illustrating a cross-flow fan quality testing device according to some embodiments of the present disclosure. Detailed Implementation

[0043] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0044] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0045] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0046] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0047] In the description of this disclosure, unless otherwise stated, "multiple" means two or more, and other quantifiers are similar; "at least one," "one or more," or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one 'a' can represent any number of 'a's; as another example, one or more of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple; "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " indicates that the preceding and following related objects are in an "or" relationship.

[0048] Although operations or steps are described in a specific order in the accompanying drawings in the embodiments of this disclosure, it should not be construed as requiring these operations or steps to be performed in the specific order or serial order shown, or requiring all of the shown operations or steps to be performed to obtain the desired result. In the embodiments of this disclosure, these operations or steps may be performed serially; they may be performed in parallel; or a portion of these operations or steps may be performed.

[0049] Cross-flow fans are a core component of air conditioners. After production, the appearance quality of the manufactured parts needs to be inspected. In related technologies, the appearance quality inspection of cross-flow fans mainly uses traditional supervised deep learning solutions and industrial vision large model few sample solutions. However, the solutions of these technologies have problems such as reliance on manually labeled samples and insufficient adaptability, which leads to a decrease in the accuracy of appearance quality inspection of the parts.

[0050] To address the aforementioned technical problems, this application proposes the following technical concept: Considering the acquisition of multi-view images of a cross-flow fan, after a single segment acquisition is completed, the control slide moves to multiple shooting positions at a preset step distance to repeatedly acquire images. After obtaining multiple segments of multi-view images, region cropping is performed to output sub-images of the fan blade area and the middle plate area. These are then input into the defect detection model corresponding to the viewpoint, and the abnormal area detection results are output and visualized. This solves the problem of missed defect detection caused by the blind spot of single-view imaging in slender rotating structures like cross-flow fans. By combining frontal and dual oblique view cameras, the circumferential surface of the fan blade segment and the middle plate areas on both sides are fully covered. The control slide acquires multi-view images in segments according to a preset step distance, ensuring complete scanning of fans of different lengths. Region cropping decomposes the large image into smaller images that fit the model input, improving detection accuracy while reducing the computational complexity of model inference.

[0051] Figure 1 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 1 ,like Figure 1 As shown, the method includes the following steps:

[0052] S11: In response to the detection trigger signal of the cross-flow fan being placed on the rotating platform, acquire multi-view images of the cross-flow fan, including images acquired by the frontal view and dual oblique view cameras, and the multi-view images cover the circumferential surface of the fan blade end and the middle plate area on both sides of the fan.

[0053] Specifically, after the upstream conveyor belt transports the cross-flow fan to the quality inspection station, the robotic arm automatically grabs the cross-flow fan and places it on the central positioning fixture of the rotating platform. After the position sensor of the rotating platform detects that the workpiece is in place, it sends a detection trigger signal to the PLC (Programmable Logic Controller) control system.

[0054] Specifically, after receiving the trigger signal, the PLC control system synchronously starts the camera and the servo motor of the rotating platform. The rotating platform drives the cross-flow fan to rotate at a preset speed. The camera synchronously performs line scanning acquisition on the circumferential surface of the cross-flow fan from three perspectives: front view, left oblique view, and right oblique view, and outputs the original image of the corresponding perspective.

[0055] Optionally, the rotational speed of the rotating platform is adaptively adjustable from 150 to 350 rad / min, the positioning accuracy is ±0.01°, and the maximum load capacity is no more than 5 kg; the resolution of a single line scan camera is 4096×5360, the frame rate is adjustable from 1000 to 2000 Hz, and the exposure time is 50 to 500 μs.

[0056] Figure 2 This is a schematic diagram of camera layout according to some embodiments of the present disclosure.

[0057] like Figure 2 As shown, the camera's field of view includes three perspectives: front view, left oblique view, and right oblique view. A front-view camera is mounted directly in front of the rotating platform, at the same horizontal level as the central axis of the cross-flow fan. It is equipped with a standard industrial lens with a focal length of 50mm and a working distance of 250mm. The lens is perpendicularly pointed to the circumferential surface of the cross-flow fan to capture images of the texture and defects on the circumferential surface of the fan blades. A first oblique-view camera is mounted diagonally above the left side of the rotating platform, at a horizontal angle of 120° with the front-view camera. It is equipped with a tilt-shift lens with a focal length of 90mm, a fixed tilt angle of 45°, and an adjustable offset of ±10mm. The working distance is 300mm, and it is used to capture images of defects on the end face of the left side split plate. A second oblique-view camera is mounted diagonally above the right side of the rotating platform, at a horizontal angle of 240° with the front-view camera. It is symmetrically distributed with the first oblique-view camera and has the same lens parameters as the first oblique-view camera. It is used to capture images of defects on the end face of the right side split plate. The light source is a strip light source with a brightness of 8000cd / m² and an illumination angle of 30°, which illuminates the surface of the cross-flow fan vertically to enhance the contrast of the fan blade texture and defects.

[0058] S12: In response to the completion of single-segment multi-view image acquisition, control the slide to move the camera module to the corresponding multiple shooting positions according to the preset step distance, and repeatedly acquire images to obtain multiple segments of multi-view images.

[0059] Specifically, after the image acquisition of the current segment is completed, the PLC control system sends a movement command to the servo driver of the linear slide. The linear slide moves along the axial direction of the cross-flow fan according to the preset step distance, and moves the camera module composed of three cameras to the shooting position of the next segment. After it is in place, the slide sends a position feedback signal to the PLC. The PLC triggers the rotating platform to rotate again and acquire images synchronously with the three cameras. The movement and acquisition process is repeated until the image acquisition of all segments is completed, and finally a complete image sequence of the cross-flow fan with multiple segments and multiple perspectives is obtained.

[0060] Optionally, the linear slide is a servo slide with a stroke of 1500mm, a movement accuracy of ±0.02mm, and a movement speed range of 50-100mm / s adjustable; the camera module is an integrated mounting bracket for a front-view camera and two oblique-view cameras, with the relative positions of the three cameras fixed and moving synchronously with the slide as a whole; the preset step distance is calculated and determined based on the axial length of the cross-flow fan and the number of segments, and the step distance is equal to the axial length of the cross-flow fan divided by the number of segments.

[0061] S13: In response to the completion of multi-segment multi-view image acquisition, perform region cropping on the multi-segment multi-view images and output the fan blade region sub-image and the mid-plate region sub-image.

[0062] Specifically, after the acquisition of multiple segments and multiple viewpoints, the image processing module loads the original images, performs feature localization on each image, identifies the positional boundaries of the fan blade region and the mid-plate region in the image, performs sliding window cropping on the two regions according to the preset sub-image size and overlap strategy, stores the cropped sub-images according to region category, and outputs the fan blade region sub-image set and the mid-plate region sub-image set.

[0063] In this embodiment, the fan blade area is the louvered cylindrical surface area of ​​the cross-flow fan body, which corresponds to the main imaging range of the front-view camera and is used to detect defects such as cracks, foreign objects such as hair, and fan blade misalignment.

[0064] In this embodiment, the center plate area is the disc-shaped connection structure area at both ends of the cross-flow fan, corresponding to the main imaging range of the two oblique-view cameras, and is used to detect defects such as glue leakage of the balance block and deformation of the center plate.

[0065] Optionally, the sub-image size is set to 720×960 pixels, and the cropping adopts an overlapping cropping strategy, with an 80-pixel overlap width between adjacent sub-images.

[0066] S14: Input the fan blade region sub-image and the center plate region sub-image into the defect detection model of the corresponding viewpoint, and output the abnormal region detection results of multiple sub-images.

[0067] Specifically, the cropped fan blade region sub-images are batch-input into the defect detection model corresponding to the frontal view, and the mid-plate region sub-images are batch-input into the defect detection model corresponding to the oblique view. After feature extraction, local feature magnification, multi-scale feature fusion and anomaly segmentation of each sub-image, the anomaly region score map of the sub-image is output. Regions with scores higher than a preset threshold are binarized to generate anomaly masks, and the coordinates of the bounding boxes of the anomaly regions are calculated. The anomaly masks and bounding boxes of all sub-images are summarized, and the anomaly region detection results of multiple sub-images are output.

[0068] In this embodiment, the defect detection model is a semi-supervised few-shot anomaly segmentation network based on meta-learning enhancement. The frontal view model and the oblique view model are trained independently. The frontal view model is optimized for defects such as cracks, hairs, and blade misalignment in the fan blade area, while the oblique view model is optimized for defects such as glue leakage and deformation in the center plate area.

[0069] In this embodiment, the abnormal region detection result includes an abnormal region mask coordinate list and an abnormal region detection box coordinate list. The mask is a pixel-level binary annotation, and the detection box is a rectangular bounding box.

[0070] S15: Draw a visualized detection result of the cross-flow fan based on the abnormal area detection results of multiple sub-maps.

[0071] Specifically, the abnormal region detection results of all sub-images are mapped back to the corresponding original multi-view image coordinate system according to the cropping position. The abnormal detection results of all sub-images on the same original image are merged. The same defect across the sub-image boundary is deduplicated and merged. The color mask and rectangular detection box of the abnormal region are superimposed on the original image, and the defect category and confidence value are labeled. A visual detection result image of the cross-flow fan with detection labels is generated and output to the human-computer interaction interface for display.

[0072] In this embodiment, mapping back to the original coordinate system is achieved by recording the cropping offset of each sub-image in the original image. The mask coordinates and detection box coordinates within the sub-image, plus the corresponding offset, can be converted into the original image coordinates.

[0073] Optionally, the color mask is filled with semi-transparent red with an opacity of 0.5, the detection box is drawn with green rectangular lines with a line width of 2 pixels, and the defect category name and confidence percentage are marked in the upper left corner of the detection box.

[0074] As can be seen from the above embodiments, by responding to the placement trigger signal to acquire multi-view images of the cross-flow fan, after the acquisition of a single segment is completed, the control slide moves to multiple shooting positions according to a preset step distance to repeatedly acquire images. After obtaining multiple segments of multi-view images, region cropping is performed to output sub-images of the fan blade area and the middle plate area. These are then input into the defect detection model of the corresponding viewpoint, and the abnormal area detection results are output and visualized. This solves the problem of missed defect detection caused by the blind spot of single-view imaging in slender rotating structures such as cross-flow fans. By combining the frontal view and dual oblique view cameras, the circumferential surface of the fan blade segment and the middle plate areas on both sides are completely covered. By controlling the slide to acquire multi-view images in segments according to a preset step distance, complete scanning of fans of different lengths is ensured. Region cropping decomposes the large image into small images that are adapted to the model input, improving the detection accuracy while reducing the computational complexity of model inference.

[0075] Figure 3 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 2 ,like Figure 3 As shown, step S12 includes the following steps:

[0076] S121: Determine the corresponding slide table movement step distance and number of segments based on the read cross-flow fan model information, and perform image acquisition operation based on the slide table movement step distance and number of segments.

[0077] Specifically, before the robotic arm grasps the cross-flow fan, the PLC control system reads the model information of the cross-flow fan to be tested from the production line system, retrieves the corresponding model parameter table based on the model information, obtains the geometric parameters of the cross-flow fan from the parameter table, calculates the slide table movement step distance and number of segments based on the axial length, and sends the calculated step distance and number of segments to the slide table servo driver and camera trigger module.

[0078] The geometric parameters include, but are not limited to, the axial length of the cross-flow fan, the length of the fan blade section, and the position of the center plate.

[0079] In this embodiment, the model information includes, but is not limited to, parameters such as the diameter of the cross-flow fan, the total axial length, the number of fan blades, and the installation position of the balance weight.

[0080] The formula for calculating the sliding step distance is as follows:

[0081] Step distance = Axial length of cross-flow fan ÷ Number of segments

[0082] The number of segments is determined based on the axial coverage of a single segment image, ensuring that each segment's imaging area covers the effective field of view of a single line scan of the camera in the axial direction, and that an 80-pixel overlap area is maintained between adjacent segments.

[0083] S122: In response to the completion of each image acquisition segment, a movement command is sent to the slide servo driver to control the slide to move the camera module to the corresponding shooting position.

[0084] Specifically, after completing the image acquisition of a segment, the camera trigger module sends a segment acquisition completion signal to the PLC control system. The PLC control system generates a movement command containing the target position coordinates and movement speed based on the current segment number and the preset calculated step distance parameters. This command is then sent to the slide servo driver according to the set communication protocol. The servo driver drives the slide motor to move the camera module along the axial guide rail to the shooting position corresponding to the next segment. After reaching the target position, the servo driver sends a position feedback signal to the PLC, waiting for the next acquisition trigger.

[0085] In this embodiment, the slide servo driver adopts a position control mode, receives absolute position commands from the PLC, and provides real-time position information through a grating ruler during movement to ensure a positioning accuracy of ±0.02mm.

[0086] In this embodiment, the content recorded by the movement command includes, but is not limited to, the absolute coordinates of the target position, the movement speed, and acceleration / deceleration curve parameters.

[0087] Optionally, the moving speed is set to 80 mm / s, and the acceleration / deceleration time is set to 0.2 s.

[0088] S123: In response to the completion of image acquisition in multiple segments, a reset command is sent to the slide servo driver to control the slide to move to the initial position.

[0089] Specifically, after the image acquisition of the last segment is completed, the camera trigger module sends a signal indicating that all acquisitions are complete to the PLC control system. The PLC control system generates a reset command and sends it to the slide servo driver. The servo driver drives the slide motor to move the camera module in the opposite direction along the axial guide rail, returning to the initial zero position. After reaching the initial position, the origin sensor is triggered, sending a reset completion signal to the PLC, and waiting for the next cross-flow fan detection process to start.

[0090] In this embodiment, the initial position is the mechanical origin of the slide table, which is physically positioned by the origin sensor, and the zero coordinates are recalibrated after each reset.

[0091] Optionally, the moving speed during the reset process is set to 100 mm / s, which is higher than the moving speed between acquisition segments.

[0092] As can be seen from the above embodiments, by reading the cross-flow fan model information to determine the corresponding slide movement step distance and number of segments, after each segment is acquired, a movement command is sent to control the slide to the next shooting position, and after the entire segment is completed, it is reset to the initial position. This achieves the adaptive and accurate positioning of the camera module for fans of different lengths and different segment structures, avoiding image misalignment or missed shots caused by positioning deviations, and providing accurate and complete original image data for image stitching and region cropping.

[0093] Figure 4 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 3 ,like Figure 4 As shown, step S13 includes the following steps:

[0094] S131: Determine the position coordinates of the fan blade region and the center plate region in multiple multi-view images.

[0095] Specifically, after loading each original image with multiple segments and multiple perspectives, a template matching-based localization algorithm is used to perform grayscale matching search between the pre-stored standard template of the fan blade edge and the contour template of the mid-plate and the original image respectively. The pixel coordinates of the upper and lower boundaries of the fan blade region and the pixel coordinates of the outer rectangle of the mid-plate region are calculated, and the position coordinates of the two regions are stored in the corresponding coordinate list.

[0096] In this embodiment, the template matching uses a normalized cross-correlation matching algorithm, and the matching threshold is set to 0.85. Regions with matching scores higher than the threshold are determined as target regions.

[0097] In this embodiment, the position coordinates of the fan blade region include the upper boundary row coordinates, lower boundary row coordinates, left boundary column coordinates, and right boundary column coordinates of the fan blade segment, which together form the circumscribed rectangle of the fan blade region.

[0098] In this embodiment, the position coordinates of the dividing plate area include the center coordinates and radius of the dividing plate disk, which are converted into the coordinates of the upper left and lower right corners of the circumscribed rectangle.

[0099] S132: Based on the position coordinates of the fan blade region and the center plate region, and using a preset sub-image size, overlap and crop the fan blade region and the center plate region, and output multiple fan blade region sub-images and center plate region sub-images.

[0100] Specifically, the cropping start point is the upper left corner of the bounding rectangle of the fan blade area. The cropping window is slid along the horizontal and vertical directions according to the preset sub-image width and height. The step size of each slide is equal to the sub-image size minus the overlap width. After traversing the entire fan blade area, all fan blade area sub-images are obtained.

[0101] Specifically, the same overlapping cropping method as the fan blade region sub-image is used to crop the bounding rectangle of the mid-section region to obtain all mid-section region sub-images. The two types of sub-images are saved and labeled with the corresponding original image number and cropping position index.

[0102] Optionally, the preset sub-image size is 720 pixels wide and 960 pixels high, the overlap width is set to 80 pixels, and the corresponding cropping sliding step size in the horizontal and vertical directions is 640 pixels and 880 pixels, respectively.

[0103] In this embodiment, the cropping process employs multi-threaded parallel processing, simultaneously cropping multiple original images.

[0104] As can be seen from the above embodiments, by determining the position coordinates of the fan blade region and the mid-plate region in multiple multi-view images, and performing overlapping cropping based on the preset sub-image size, multiple fan blade region sub-images and mid-plate region sub-images are output. The key regions scattered in the large line scan image are accurately extracted into standardized sub-images. Overlapping cropping ensures that defects such as cracks and hairs located at the cropping boundary are not truncated, and the defect information is completely preserved, reducing the input size and computational overhead of the model inference.

[0105] Figure 5 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 4 ,like Figure 5 As shown, the method also includes the following steps:

[0106] S51: In response to the received model training instruction, load unlabeled good sample images and weakly labeled bad sample images.

[0107] Specifically, upon receiving the model training instruction from the user, the training framework loads the dataset of unlabeled good sample images and the dataset of weakly labeled bad sample images from the specified data directory. It reads and verifies the format of the two types of samples, parses the defect category labels and rough location box information attached to the weakly labeled bad samples, organizes the sample images and corresponding labels into a standardized training data format, and stores them in the training data cache area.

[0108] In this embodiment, the unlabeled good sample image is a cut-out image of a normal, defect-free cross-flow fan, without any accompanying labeling information.

[0109] In this embodiment, the weakly labeled defective sample image is a cut-out image of a cross-flow fan with defects, including four types of defects: cracks, foreign objects such as hair, glue leakage from the balance block, and skewed fan blades. The labeling content includes the defect category name and a rough rectangular location box.

[0110] Optionally, the accuracy of the weakly labeled bounding box is allowed to have a deviation of ±20 pixels.

[0111] S52: Perform data augmentation on weakly labeled defective sample images to generate an augmented defective sample set.

[0112] Specifically, multiple lightweight data augmentation operations are sequentially performed on each loaded weakly labeled defective sample image, and the corresponding weakly labeled location boxes are synchronously transformed. All augmented samples are then merged with the original weakly labeled samples to form an augmented defective sample set.

[0113] Data augmentation operations include, but are not limited to, random rotation, random scaling, horizontal mirroring, and brightness fine-tuning.

[0114] Optionally, the random rotation angle range is ±10°, the random scaling ratio range is 0.8 to 1.2 times, the probability of horizontal mirror flip is 50%, the brightness fine-tuning range is ±10%, and the enhancement strategies are all geometric transformation and photometric transformation.

[0115] S53: Take the unlabeled good sample images and the enhanced defective sample set as input, perform iterative training of the model in the meta-training stage, and output the meta-trained base model.

[0116] Specifically, the unlabeled good samples and the enhanced defective sample set are randomly divided into a meta-training set and a meta-validation set according to the set ratio. The parameters of the defect detection network are initialized, and after setting the training hyperparameters, multiple rounds of iterative training are started. Before each round of iteration, the support set and query set are randomly sampled from the training set. The combined loss function value is calculated by forward propagation, and the network parameters are updated by backpropagation. After each round of training, the validation loss is calculated on the validation set. The convergence status of the model is monitored according to the validation loss. After the preset stopping condition is reached, the optimal model parameters are saved, and the meta-training basic model is output.

[0117] In this embodiment, the meta-training stage adopts the MAML (Model-Agnostic Meta-Learning) framework. The training objective is to obtain a set of network parameters with good generalization initialization capabilities, so that the model can quickly adapt to new fan models with a small number of samples.

[0118] In this embodiment, the combined loss function is composed of three weighted parts: contrast loss, pseudo-label loss, and cross-view consistency loss; the meta-training base model includes all parameters of the feature extraction backbone network, the local feature amplification module, and the multi-scale feature fusion layer.

[0119] S54: In response to the received fan adaptation command, load the meta-training base model and unlabeled good samples of the fan to be adapted, perform parameter adjustment operations in the adaptation stage, and output the adapted defect detection model.

[0120] Specifically, upon receiving the adaptation instruction for a new model of cross-flow fan, the meta-trained basic model is loaded as initialization parameters, a small number of unlabeled good sample images of the model to be adapted are loaded, all parameters of the feature extraction backbone network are frozen, the parameters of the meta-learning adaptation head are fine-tuned and trained, and after gradient descent iteration, the adaptation head parameters converge. All parameters of the backbone network and the adaptation head are saved, and the defect detection model adapted for the fan model is output.

[0121] In this embodiment, the fan adaptation command includes the fan model to be adapted and the corresponding unmarked good sample path.

[0122] As can be seen from the above embodiments, by loading unlabeled good sample images and weakly labeled bad sample images, performing data augmentation on the bad samples, and then inputting both types of samples together to perform iterative training of the model in the meta-training stage, the meta-trained basic model is output. When adapting to a new model, the basic model and unlabeled good samples are loaded to complete the rapid adaptation. By utilizing the general feature extraction capability of meta-training and the rapid fine-tuning mechanism in the adaptation stage, the adaptation of the new model fan can be carried out without interrupting the production line, thus shortening the project introduction cycle.

[0123] Figure 6 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 5 ,like Figure 6 As shown, step S53 includes:

[0124] S531: In response to each training iteration, determine multiple loss values ​​for each training iteration.

[0125] Specifically, during the forward propagation process of each training iteration, the query set samples are input into the network to obtain the predicted output, and the three losses—contrast loss, pseudo-label loss, and cross-view consistency loss—are calculated respectively.

[0126] Among them, the contrast loss is calculated based on the feature distance between unlabeled good samples and weakly labeled bad samples, the pseudo-label loss is calculated based on the normal pseudo-labels automatically generated from unlabeled good samples and the prediction results, and the cross-view consistency loss is calculated based on the feature similarity of images of the same component from different viewpoints.

[0127] In this embodiment, the contrast loss adopts the form of triplet loss, which brings the feature distance of similar samples closer and pushes the feature distance of dissimilar samples further apart, thereby strengthening the feature boundary between normal and abnormal regions; the pseudo-label loss automatically marks unlabeled good samples as fully normal, and calculates the cross-entropy loss with the abnormal score map predicted by the model, thereby expanding the supervision signal; the cross-view consistency loss calculates the cosine distance of the feature vectors of the same region of the same cross-flow fan under the frontal and oblique viewing angles, so as to keep the cross-view features of the normal region consistent.

[0128] S532: The total loss value is obtained by weighting and summing multiple loss values ​​according to their preset weights.

[0129] Specifically, the comparison loss value is multiplied by the comparison loss weight, the pseudo-label loss value is multiplied by the pseudo-label loss weight, and the cross-view consistency loss value is multiplied by the cross-view consistency loss weight. The three weighted loss values ​​are then added together to obtain the total loss value for this iteration. The total loss value is used as the optimization target for backpropagation.

[0130] Optionally, the preset weights for the contrast loss and pseudo-label loss are 0.4, 0.3, and 0.3 respectively, and the sum of the three weights is 1.

[0131] S533: Update the model parameters using backpropagation based on the total loss value.

[0132] Specifically, with the total loss value as the optimization objective, the gradient value of each parameter is calculated layer by layer in the reverse direction starting from the network output layer. The AdamW optimizer is used to update the backbone parameters of the feature extraction module, the parameters of the local feature amplification module, and the parameters of the multi-scale feature fusion layer of the network based on the gradient value and the current learning rate. The parameters of the meta-learning adapter head are updated synchronously. After the update is completed, the next training iteration begins. The learning rate adopts a cosine annealing scheduling strategy to dynamically decay with each iteration.

[0133] In this embodiment, the weight decay coefficient of the AdamW optimizer is set to 0.0001, the cosine annealing learning rate scheduling is set to the initial learning rate of 0.0001 as the maximum value, and the learning rate is gradually reduced according to the cosine curve with each iteration. Backpropagation is implemented using an automatic differentiation mechanism, and the gradient pruning threshold is set to 1.0 to prevent gradient explosion.

[0134] S534: In response to the failure of the validation set loss value to decrease after a series of preset rounds, stop model training, save the updated target model parameters, and output the meta-training base model.

[0135] Specifically, after completing the training for a preset interval of rounds, the current model is forward-inferenced on the meta-validation set to calculate the total validation loss value. The current validation loss is compared with the historical lowest validation loss. If the validation loss does not decrease, the continuous non-decreasing counter is incremented by 1. If the validation loss becomes the lowest value, the counter is reset and the current model is saved as the optimal model. When the continuous non-decreasing counter reaches the preset number of rounds, the early stopping mechanism is triggered to terminate the training process. The saved optimal model parameters are output as the meta-training base model.

[0136] Optionally, the preset verification interval is to perform verification once every 10 rounds of training, and the preset number of consecutive rounds is set to 20 rounds, that is, if the verification loss does not decrease in 20 consecutive rounds, training is stopped.

[0137] In this embodiment, the meta-training base model is saved as a weight file, including but not limited to all parameters of the feature extraction backbone, CBAM attention module, local feature amplification module, multi-scale feature fusion layer, and segmentation output layer.

[0138] As can be seen from the above embodiments, multiple loss values ​​are calculated in each training iteration, and the total loss value is obtained by weighted summation according to preset weights. The model parameters are updated by backpropagation. When the loss on the validation set does not decrease for a consecutive preset number of rounds, an early stopping mechanism is triggered. The boundary between normal and abnormal features is strengthened by comparing the loss, the utilization efficiency of unlabeled data is improved by using pseudo-label loss, and the feature alignment of the same part under different perspectives is forced by cross-view consistency loss. The early stopping mechanism avoids invalid iterations and overfitting, and improves the training efficiency and generalization ability of the model under conditions of few samples.

[0139] Figure 7 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 6 ,like Figure 7 As shown, step S14 includes:

[0140] S141: Input the fan blade region sub-image and the mid-plate region sub-image into the feature extraction network, and output feature maps of multiple resolutions.

[0141] Specifically, the fan-blade region sub-image and the mid-plate region sub-image are input into the feature extraction backbone network. The sub-images are sequentially processed through multiple layers of deep separable convolutional and inverse residual structures for downsampling feature extraction. Feature maps of corresponding scales are extracted from layers 2 to 5 of the network, resulting in feature maps with resolutions of the original... Figure 1 The feature maps are divided into four levels: 2, 1 / 4, 1 / 8, and 1 / 16. At the same time, dual weighting of channel attention and spatial attention is applied to the feature maps from the 3rd to the 5th layers.

[0142] Optionally, the feature extraction network can be designed to be lightweight using depthwise separable convolutions and inverse residual structures. The feature maps at multiple resolutions are as follows: 1 / 2 resolution feature map with a size of 360×480 and 24 channels; 1 / 4 resolution feature map with a size of 180×240 and 40 channels; 1 / 8 resolution feature map with a size of 90×120 and 112 channels; and 1 / 16 resolution feature map with a size of 45×60 and 160 channels.

[0143] Optionally, the dual attention module is a CBAM (Convolutional Block Attention Module), which sequentially filters key feature channels through channel attention and locates the spatial position of defects through spatial attention.

[0144] S142: Perform weighted amplification processing on the features of the fan blade edge and the area surrounding the center plate based on feature maps of multiple resolutions.

[0145] Specifically, the feature maps of multiple resolutions output by the feature extraction network are input into the local feature amplification module. Based on the distribution mask of high defect probability areas around the fan blade edge and the middle plate obtained from the pre-training stage, the module applies an adaptive weight amplification of 1.5 to 2.0 times to the feature channels of the corresponding regions in each resolution feature map. At the same time, the receptive field of the feature map is expanded by using a dilated convolution feature pyramid structure.

[0146] In a convolutional neural network, the receptive field is the size of the region on the input image that is mapped by the pixels on the feature map output by each layer of the convolutional neural network.

[0147] Optionally, the local feature magnification module uses two levels of dilated convolutions with dilation rates of 2 and 4 to build a feature pyramid structure, expanding the receptive field to 4 times that of the original 3×3 convolution.

[0148] Among them, the high defect probability region is the fan blade edge strip region and the annular region around the middle plate, which are derived from a large number of sample statistics and are assigned higher feature weights.

[0149] S143: Perform weighted fusion on the feature maps of multiple resolutions after feature magnification, and output the fused feature map.

[0150] Specifically, the four resolution feature maps after local feature amplification are input into the multi-scale feature fusion layer. The low-resolution feature map is upsampled and interpolated to gradually increase the resolution to the size of the high-resolution feature map. During the upsampling process, a coordinated attention mechanism is introduced to automatically calibrate the channel distribution deviation of features at different levels. The four levels of features are weighted and summed pixel by pixel according to the ratio of 0.7 for high-resolution features and 0.3 for low-resolution features to obtain the fused feature map.

[0151] Optionally, the weighting strategy for the four resolution feature maps is as follows: the high-resolution feature maps at 1 / 2 and 1 / 4 resolutions are assigned a total weight of 0.7 to capture the detailed textures of defects such as cracks and hairs; the low-resolution feature maps at 1 / 8 and 1 / 16 resolutions are assigned a total weight of 0.3 to capture the overall structural integrity and large-scale deformation features of the fan blades.

[0152] In this embodiment, the coordinated attention mechanism encodes position information and channel relationships by performing global average pooling along the horizontal and vertical directions, automatically calibrating the category distribution deviation caused by small image cropping, and using bilinear interpolation for upsampling.

[0153] S144: Output an anomaly region score map based on the fused feature map.

[0154] Specifically, the fused feature map is input into the segmentation output layer, and the number of channels is compressed to 1 through a 1×1 convolution. The output value of each pixel is mapped to the range of 0 to 1 through the Sigmoid activation function to obtain an anomaly region score map of the same size as the fused feature map.

[0155] In the score image, the value of each pixel represents the probability that the location belongs to an abnormal area. The closer the value is to 1, the higher the probability that the location is a defect. The closer the value is to 0, the higher the probability that the location is a normal area.

[0156] In this embodiment, the segmentation output layer consists of two convolutional layers: the first layer is a 3×3 convolution for feature integration, and the second layer is a 1×1 convolution for channel dimensionality reduction, outputting a single-channel score map.

[0157] In this embodiment, the resolution of the abnormal region score map is the resolution of the fused feature map, which is half the size of the input sub-map. In the post-processing stage, the original sub-map size is restored by interpolation.

[0158] S145: In response to an abnormal region score exceeding a preset threshold, mark the abnormal region and output the mask coordinates and detection box coordinates of the abnormal region.

[0159] Specifically, the abnormal region score map is upsampled to the same resolution as the original input sub-image. The score value is compared with a preset threshold pixel by pixel. Pixels with scores higher than the preset threshold are marked as abnormal pixels. A binarized abnormal region mask is generated. Connected component analysis is performed on the connected abnormal pixel regions to extract the contour of each independent abnormal region. The minimum bounding rectangle of each connected component is calculated as the abnormal detection box. The mask coordinate set and the detection box coordinate set of all abnormal regions are summarized, and the abnormal region detection result of the sub-image is output.

[0160] In this embodiment, the preset threshold is determined by Youden's index, which means searching on the validation set for the score that minimizes the sum of the false negative rate and the false positive rate as the optimal threshold.

[0161] Optionally, the threshold for the front-view fan blade model is set to 0.65, and the threshold for the oblique-view split plate model is set to 0.6.

[0162] In this embodiment, the mask coordinates are the set of row and column coordinates of abnormal pixels in the subgraph coordinate system, and the detection box coordinates are the rectangle formed by the upper left and lower right corner coordinates of each abnormal connected component. The connected component analysis adopts the 8-neighborhood connectivity criterion, and at the same time filters out tiny connected components with an area of ​​less than 10 pixels to reduce noise false detections.

[0163] As can be seen from the above embodiments, by inputting the sub-image into the feature extraction network and outputting feature maps of multiple resolutions, the features of the fan blade edge and the surrounding area of ​​the middle plate are weighted and amplified. The amplified multi-resolution feature maps are weighted and fused to output a fused feature map, and an abnormal region score map is output. The abnormal region is marked according to the threshold and the mask coordinates and detection box coordinates are output. The network structure is optimized for local micro-defects such as micro-cracks and hairs in the cross-flow fan. The receptive field is expanded by using a dilated convolutional pyramid, which reduces the false negative rate of micro-defects.

[0164] Figure 8 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 7 ,like Figure 8 As shown, the method also includes the following steps:

[0165] S81: In response to the detection of defects in the cross-flow fan, identify the defect categories and statistically analyze the percentage of each type of defect.

[0166] Specifically, when a defect is detected in a single cross-flow fan, the defects are classified and summarized according to the category label of each defect record. At the same time, the historical detection data of cross-flow fans on the same production line within the historical period are retrieved, and the total number of occurrences of four types of defects, namely cracks, hair and foreign objects, glue leakage of balance block and fan blade misalignment, are counted. The percentage of each type of defect in the total number of defects is calculated, as well as the defect rate of each type of defect, and the defect category distribution statistics are generated.

[0167] In this embodiment, the defect categories include four types: cracks, foreign objects such as hair, glue leakage from the balance block, and fan blade misalignment.

[0168] S82: Based on the defect category and percentage data, query the defect process parameter mapping library and output the corresponding process adjustment strategy information.

[0169] Specifically, the statistically obtained defect percentage data is input into the defect process parameter mapping library. The mapping library matches the corresponding process impact factors according to the percentage weight of each type of defect, retrieves the upstream process parameters and adjustment directions corresponding to each type of defect, generates a priority-ranked list of process adjustment strategies by integrating the adjustment needs of multiple defect types, and outputs structured process adjustment strategy information.

[0170] In this embodiment, the process adjustment strategy records information including but not limited to process parameter names, current reference values, suggested adjustment values, adjustment ranges, and expected improvement effects.

[0171] In this embodiment, the defect process parameter mapping library is a pre-built knowledge base that stores the causal mapping relationship between defect types and process parameters such as injection molding, assembly, and gluing.

[0172] Among them, the higher the weight of a defect type, the higher the priority of the corresponding process adjustment strategy.

[0173] S83: Send process adjustment strategy information to the production line system and display it on the process monitoring interface.

[0174] Specifically, the generated process adjustment strategy information is sent to the production line system via the industrial Ethernet protocol. After receiving the strategy information, the production line system parses the format and pushes the strategy content to the early warning area of ​​the workshop process monitoring interface. The priority, parameter name and suggested value of each adjustment strategy are displayed in the form of a highlighted card. After confirmation, it can be sent to the corresponding process equipment to perform parameter adjustment, and the adjustment record is stored in the process optimization log.

[0175] In this embodiment, the strategy cards are arranged in descending order of priority.

[0176] In this embodiment, the process optimization log records, including but not limited to, adjustment time, adjustment parameters, and a comparison of defect rates before and after the adjustment.

[0177] As can be seen from the above embodiments, by adding an intelligent closed-loop linkage mechanism from the detection results to the process adjustment after the defect detection is completed, when a defect is detected in the cross-flow fan, the defect category is identified and the proportion data of each type of defect is statistically analyzed. The defect process parameter mapping library is queried, the corresponding process adjustment strategy information is output, sent to the production line system and displayed on the process monitoring interface, thereby reducing the recurrence rate of the same type of defect.

[0178] Figure 9 This is a flowchart illustrating a cross-flow fan quality testing method according to some embodiments of the present disclosure. Figure 8 ,like Figure 9 As shown, step S15 includes:

[0179] S151: In response to the same defect region being detected as an anomaly from multiple viewpoints, determine the feature similarity of the defect region from multiple viewpoints.

[0180] Specifically, a spatial coordinate mapping relationship is established between the three viewpoint images. The three-dimensional coordinates of the abnormal area detected by each viewpoint are projected onto the image plane of other viewpoints. The existence of a corresponding abnormal area around the projection position is searched. If the defect at the same physical location is detected as abnormal in two or three viewpoints, the depth feature vector of the defect area in each viewpoint is extracted, and the cosine similarity between the feature vectors is calculated as the feature similarity of the defect area in multiple viewpoints.

[0181] In this embodiment, the spatial coordinate mapping relationship is obtained through camera calibration, which includes the intrinsic and extrinsic parameter matrices of the three cameras.

[0182] In this embodiment, the deep feature vector is extracted from the multi-scale fusion feature layer of the defect detection model, with a dimension of 256. The cosine similarity value ranges from -1 to 1. The closer the value is to 1, the more similar the two feature vectors are, and the higher the probability that the corresponding defects are the same physical defects.

[0183] S152: In response to a feature similarity exceeding a preset threshold, merge the detection results from multiple perspectives into a single defect record.

[0184] Specifically, the calculated multi-view feature similarity is compared with a preset similarity threshold. If the feature similarity is higher than the preset threshold, it is determined that the abnormal regions detected by multiple views correspond to the same physical defect. The detection boxes and masks of multiple views are merged, and the union of the detection boxes of each view is taken as the merged detection box. The union of the merged mask projections of each view is taken as the merged mask. A merged defect record containing multi-view information is generated, and the views from which the defect was detected are recorded. If the feature similarity is lower than the preset threshold, it is determined to be a different defect, and its respective defect record is retained.

[0185] Optionally, the preset similarity threshold is set to 0.75.

[0186] In this embodiment, the defect record includes, but is not limited to, the unique defect number, defect category, confidence level, merged mask coordinates, merged detection box coordinates, list of detected viewpoints, and the segment number field.

[0187] S153: Draw the merged defect mask and detection box on the original image based on the defect record, and generate a visual detection result.

[0188] Specifically, the merged defect records are traversed. For each defect record, the merged mask is drawn on the original image from the corresponding viewpoint with a semi-transparent color fill. Different defect categories are distinguished by different colors. The merged detection box is drawn as a rectangular line around the mask, and the defect number, category, and confidence level text are labeled next to the detection box.

[0189] Specifically, after the defects are drawn, an overview information bar is added to the top of the image to generate a complete multi-view visualization image of the cross-flow fan inspection results.

[0190] The overall inspection information bar records information including, but not limited to, the total number of defects, the number of defects of each type, and the inspection time.

[0191] Optionally, crack defects are marked with a red mask, hair defects with a blue mask, glue leakage defects with a yellow mask, and fan blade misalignment defects with a purple mask. The mask transparency is uniformly set to 0.5. The detection box is drawn with a bright green solid line with a line width of 2 pixels. The detection overview information bar is located at the top of the image with a height of 40 pixels. The background is dark gray, and the text is white.

[0192] As can be seen from the above embodiments, when the same defect area is detected as abnormal from multiple perspectives, the feature similarity of the defect area under multiple perspectives is calculated. When the similarity is higher than a preset threshold, the detection results of multiple perspectives are merged into the same defect record, and the merged defect mask and detection box are drawn on the original image. This solves the problem of redundant records caused by repeated detection of the same physical defect in multiple perspective images. The feature similarity check avoids erroneous merging and improves the accuracy of the detection report.

[0193] Figure 10 This is a block diagram illustrating a cross-flow fan quality testing device according to some embodiments of the present disclosure. (Refer to...) Figure 10 The device includes: a data acquisition module 1001, a control module 1002, a first output module 1003, a second output module 1004, and a drawing module 1005.

[0194] The acquisition module 1001 is used to acquire multi-view images of the cross-flow fan in response to the detection trigger signal of the cross-flow fan being placed on the rotating platform. The multi-view images include images acquired by the frontal view and dual oblique view cameras, and the multi-view images cover the circumferential surface of the fan blade end and the middle plate area on both sides of the fan.

[0195] The control module 1002 is used to control the slide to move the camera module to multiple corresponding shooting positions according to the preset step distance in response to the completion of single-segment multi-view image acquisition, and repeatedly acquire images to obtain multiple segments of multi-view images.

[0196] The first output module 1003 is used to respond to the completion of multi-segment multi-view image acquisition, perform region cropping on the multi-segment multi-view images, and output the fan blade region sub-image and the center plate region sub-image.

[0197] The second output module 1004 is used to input the fan blade region sub-image and the middle plate region sub-image into the defect detection model of the corresponding viewpoint, and output the abnormal region detection results of multiple sub-images.

[0198] The drawing module 1005 is used to draw the visual detection results of the cross-flow fan based on the abnormal area detection results of multiple sub-graphs.

[0199] In one embodiment of this application, the control module 1002 includes:

[0200] The first determining unit is used to determine the corresponding slide table movement step distance and number of segments based on the read cross-flow fan model information, and to perform image acquisition operation based on the slide table movement step distance and number of segments.

[0201] The first transmitting unit is used to send a movement command to the slide servo driver in response to the completion of each image acquisition segment, so as to control the slide to move the camera module to the corresponding shooting position.

[0202] The second transmitting unit is used to send a reset command to the slide servo driver in response to the completion of image acquisition in multiple segments, and control the slide to move to the initial position.

[0203] In one embodiment of this application, the first output module 1003 includes:

[0204] The second determining unit is used to determine the position coordinates of the fan blade region and the center plate region in multiple multi-view images.

[0205] The cropping unit is used to overlap and crop the fan blade region and the middle dividing plate region based on the position coordinates of the fan blade region and the position coordinates of the middle dividing plate region according to the preset sub-image size, and output multiple fan blade region sub-images and middle dividing plate region sub-images.

[0206] In one embodiment of this application, the cross-flow fan quality detection device further includes:

[0207] The loading module is used to load unlabeled good sample images and weakly labeled poor sample images in response to received model training instructions.

[0208] The data augmentation module is used to perform data augmentation processing on weakly labeled defective sample images to generate an augmented defective sample set.

[0209] The training module takes unlabeled good sample images and enhanced defective sample sets as input, performs iterative training of the model in the meta-training stage, and outputs the meta-trained base model.

[0210] The adjustment module is used to respond to the received fan adaptation command, load the meta-trained basic model and unlabeled good samples of the fan to be adapted, perform parameter adjustment operations in the adaptation stage, and output the adapted defect detection model.

[0211] In one embodiment of this application, the training module includes:

[0212] The third determining unit is used to determine multiple loss values ​​for each training iteration in response to each training iteration.

[0213] The first generation unit is used to obtain the total loss value by weighted summation of multiple loss values ​​according to preset weights.

[0214] The update unit is used to update the model parameters based on backpropagation of the total loss value.

[0215] The first output unit is used to stop model training and save the updated target model parameters in response to the failure of the validation set loss value to decrease for a consecutive preset number of rounds, and output the meta-training base model.

[0216] In one embodiment of this application, the second output module 1004 includes:

[0217] The second output unit is used to input the fan blade region sub-image and the mid-plate region sub-image into the feature extraction network and output feature maps of multiple resolutions.

[0218] The weighted amplification unit is used to perform weighted amplification processing on the features of the fan blade edge and the surrounding area of ​​the split plate based on feature maps of multiple resolutions.

[0219] The weighted fusion unit is used to perform weighted fusion on feature maps of multiple resolutions after feature magnification, and output a fused feature map.

[0220] The third output unit is used to output an anomaly region score map based on the fused feature map.

[0221] The labeling unit is used to label abnormal regions in response to an abnormal region score exceeding a preset threshold, and outputs the mask coordinates and detection box coordinates of the abnormal region.

[0222] In one embodiment of this application, the cross-flow fan quality detection device further includes:

[0223] The identification module is used to detect defects in the cross-flow fan, identify the defect categories, and statistically analyze the percentage of each type of defect.

[0224] The query module is used to query the defect process parameter mapping library based on defect category and percentage data, and output the corresponding process adjustment strategy information.

[0225] The sending module is used to send process adjustment strategy information to the production line system and display it on the process monitoring interface.

[0226] In one embodiment of this application, the drawing module 1005 includes:

[0227] The fourth determining unit is used to determine the feature similarity of the defect region under multiple viewpoints in response to the same defect region being detected as an anomaly under multiple viewpoints.

[0228] The merging unit is used to merge the detection results from multiple perspectives into a single defect record in response to a feature similarity exceeding a preset threshold.

[0229] The second generation unit is used to draw the merged defect mask and detection box on the original image based on the defect record, and generate a visual detection result.

[0230] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0231] Figure 11 This is a physical block diagram illustrating a cross-flow fan quality testing device according to some embodiments of the present disclosure. Figure 11 As shown, it includes:

[0232] Processor 1101;

[0233] Memory 1102 is used to store processor-executable instructions;

[0234] The processor 1102 is configured to execute the cross-flow fan quality detection method shown in the above method embodiment.

[0235] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method. The implementation principle and technical effect are similar, and will not be elaborated in detail here.

[0236] Figure 12 This is a block diagram illustrating a cross-flow fan quality testing device according to some embodiments of the present disclosure.

[0237] Reference Figure 12 The device 1200 may include one or more of the following components: a processing component 1202, a memory 1204, a power component 1206, a multimedia component 1208, an audio component 1210, an input / output (I / O) interface 1212, a sensor component 1214, and a communication component 1216.

[0238] Processing component 1202 typically controls the overall operation of device 1200, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1202 may include one or more processors 1220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1202 may include one or more modules to facilitate interaction between processing component 1202 and other components. For example, processing component 1202 may include a multimedia module to facilitate interaction between multimedia component 1208 and processing component 1202.

[0239] Memory 1204 is configured to store various types of data to support the operation of device 1200. Examples of this data include instructions for any application or method operating on device 1200, contact data, phonebook data, messages, pictures, videos, etc. Memory 1204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0240] The power supply component 1206 provides power to the various components of the device 1200. The power supply component 1206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 1200.

[0241] Multimedia component 1208 includes a screen that provides an output interface between device 1200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1208 includes a front-facing camera and / or a rear-facing camera. When device 1200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0242] Audio component 1210 is configured to output and / or input audio signals. For example, audio component 1210 includes a microphone (MIC) configured to receive external audio signals when device 1200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1204 or transmitted via communication component 1216. In some embodiments, audio component 1210 also includes a speaker for outputting audio signals.

[0243] I / O interface 1212 provides an interface between processing component 1202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0244] Sensor assembly 1214 includes one or more sensors for providing status assessments of various aspects of device 1200. For example, sensor assembly 1214 may detect the on / off state of device 1200, the relative positioning of components such as the display and keypad of device 1200, changes in the position of device 1200 or a component of device 1200, the presence or absence of user contact with device 1200, the orientation or acceleration / deceleration of device 1200, and temperature changes of device 1200. Sensor assembly 1214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0245] Communication component 1216 is configured to facilitate wired or wireless communication between device 1200 and other devices. Device 1200 can access wireless networks based on communication standards, such as WiFi, 3G, 4G, 5G, other communication standards, or combinations thereof. In some embodiments of this disclosure, communication component 1216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of this disclosure, communication component 1216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0246] In some embodiments of this disclosure, the apparatus 1200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0247] In some embodiments of this disclosure, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1204 including instructions, which can be executed by a processor 1220 of the device 1200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0248] This disclosure provides a non-transitory computer-readable storage medium that, when executed by a processor, enables the processor to perform the cross-flow fan quality detection method shown in the above-described method embodiments.

[0249] The non-transitory computer-readable storage medium provided in this disclosure has been described in detail in the embodiments of the relevant method, and its implementation principle and technical effect are similar, so it will not be described in detail here.

[0250] This disclosure provides a computer program product, including:

[0251] A computer program, when executed by a processor, implements the cross-flow fan quality detection method shown in the above method embodiments.

[0252] The specific manner in which the computer program product provided in this disclosure performs its operation has been described in detail in the embodiments related to the method. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0253] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.

[0254] It should be understood that, unless otherwise expressly specified and limited, the terms "joining," "attaching," "installing," "connecting," "linking," "fixing," etc., used in the embodiments of this disclosure should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms herein based on the specific circumstances.

[0255] Furthermore, the term "above" as used herein with respect to components, elements, or material layers formed or located "above" a surface may be used to indicate that the component, element, or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements, or layers are arranged between the surface and the component, element, or material layer. However, the term "above" as used with respect to components, elements, or material layers formed or located "above" a surface may also optionally have a specific meaning: that the component, element, or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, for example, in direct contact with the surface.

[0256] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0257] It should be understood that spatial relative terms, such as “above,” “upper,” “below,” and “lower,” are used herein to describe the relationship between one element and another shown in the figures. In addition to the orientation depicted in the figures, these spatial relative terms are also intended to encompass different orientations of the device in use or operation. For example, if the device in the figures is flipped, an element described as “above” or “upper” relative to another element would be “below” or “lower” relative to that other element. Thus, depending on the spatial orientation of the device, the term “above” encompasses both above and below orientations. Devices may have other orientations (e.g., rotated 90 degrees or in other orientations), and the spatial relative terms used herein should be interpreted accordingly.

[0258] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0259] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0260] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0261] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for quality inspection of a cross-flow fan, characterized in that, include: In response to a detection trigger signal that the cross-flow fan is placed on a rotating platform, a multi-view image of the cross-flow fan is acquired, wherein the multi-view image includes images acquired by a frontal view camera and a dual oblique view camera, and the multi-view image covers the circumferential surface of the fan blade tip and the split plate area on both sides of the fan. In response to the completion of single-segment multi-view image acquisition, the control slide moves the camera module to multiple corresponding shooting positions according to the preset step distance, and repeatedly acquires images to obtain multiple segments of multi-view images. In response to the completion of the acquisition of the multi-segment multi-view images, the multi-segment multi-view images are cropped to output a fan blade region sub-image and a mid-plate region sub-image. The fan blade region sub-image and the center plate region sub-image are respectively input into the defect detection model of the corresponding viewpoint, and the abnormal region detection results of multiple sub-images are output. The visualization results of the cross-flow fan are drawn based on the abnormal region detection results of the multiple sub-graphs.

2. The cross-flow fan quality testing method according to claim 1, characterized in that, The control slide moves the camera module to multiple corresponding shooting positions according to a preset step distance, including: The corresponding slide table movement step distance and number of segments are determined based on the read cross-flow fan model information, and an image acquisition operation is performed based on the slide table movement step distance and number of segments. In response to the completion of each image acquisition, a movement command is sent to the slide servo driver to control the slide to move the camera module to the corresponding shooting position; In response to the completion of image acquisition in multiple segments, a reset command is sent to the slide servo driver to control the slide to move to the initial position.

3. The cross-flow fan quality testing method according to claim 1, characterized in that, The step of cropping the multi-segment, multi-view images to output a fan-blade region sub-image and a mid-plate region sub-image includes: Determine the position coordinates of the fan blade region and the mid-plate region in multiple segments and multi-view images; Based on the position coordinates of the fan blade region and the center plate region, and using a preset sub-image size, the fan blade region and the center plate region are overlapped and cropped to output multiple fan blade region sub-images and center plate region sub-images.

4. The cross-flow fan quality testing method according to claim 1, characterized in that, The method further includes: In response to the received model training instructions, load unlabeled good sample images and weakly labeled bad sample images; Data augmentation processing is performed on the weakly labeled defective sample images to generate an enhanced defective sample set; The unlabeled good sample images and the enhanced defective sample set are used as input to perform iterative model training in the meta-training stage, and the meta-training base model is output. In response to the received fan adaptation command, the meta-training base model and unlabeled good samples of the fan to be adapted are loaded, the parameter adjustment operation in the adaptation stage is performed, and the adapted defect detection model is output.

5. The cross-flow fan quality testing method according to claim 4, characterized in that, The iterative training of the model during the meta-training phase outputs a meta-trained base model, including: In response to each training iteration, determine multiple loss values ​​for each training iteration; The total loss value is obtained by weighting and summing the multiple loss values ​​according to their preset weights. The parameters of the model are updated by backpropagation based on the total loss value; If the loss value on the validation set does not decrease after a set number of consecutive rounds, stop training the model, save the updated target model parameters, and output the meta-training base model.

6. The cross-flow fan quality testing method according to claim 1, characterized in that, The step involves inputting the fan blade region sub-image and the center plate region sub-image into a defect detection model with corresponding viewpoints, and outputting abnormal region detection results for multiple sub-images, including: Input the fan blade region sub-image and the mid-plate region sub-image into the feature extraction network to output feature maps of multiple resolutions; The features of the fan blade edge and the area surrounding the center plate are weighted and magnified based on the feature maps of the multiple resolutions; The feature maps at multiple resolutions after feature magnification are weighted and fused to output a fused feature map. Output an anomaly region score map based on the fused feature map; When an abnormal region score exceeds a preset threshold, the abnormal region is marked, and the mask coordinates and detection box coordinates of the abnormal region are output.

7. The cross-flow fan quality testing method according to claim 1, characterized in that, After drawing the visualized detection results of the cross-flow fan based on the abnormal region detection results of the multiple sub-graphs, the method further includes: In response to the detection of defects in the cross-flow fan, the defect categories are identified and the percentage data of each type of defect are statistically analyzed; Based on the defect category and the percentage data, query the defect process parameter mapping library and output the corresponding process adjustment strategy information; The process adjustment strategy information is sent to the production line system and displayed on the process monitoring interface.

8. The cross-flow fan quality testing method according to claim 1, characterized in that, The step of drawing the visualized detection results of the cross-flow fan based on the abnormal region detection results of the multiple sub-graphs includes: In response to the same defect region being detected as an anomaly from multiple viewpoints, the feature similarity of the defect region under multiple viewpoints is determined. In response to a feature similarity exceeding a preset threshold, the detection results from multiple perspectives are merged into a single defect record. Based on the defect records, a merged defect mask and detection box are drawn on the original image to generate a visual detection result.

9. A cross-flow fan quality testing device, characterized in that, include: The device comprises a data acquisition module, a control module, a first output module, a second output module, and a drawing module; wherein the cross-flow fan quality detection device is configured to perform the cross-flow fan quality detection method as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to perform the cross-flow fan quality detection method as described in any one of claims 1 to 8.

11. A computer program product, characterized in that, include: A computer program, which, when executed by a processor, implements the cross-flow fan quality detection method as described in any one of claims 1 to 8.

Images