Part hollowing rate determination method, device, equipment, medium and program product

Abstract

The application relates to a part hollowing rate determination method, device, equipment, medium and program product. The method comprises the following steps: inputting image data and a preset prompt word into a part recognition model which has been pre-trained to obtain a target bounding box output by the part recognition model; the target bounding box marks an area occupied by the part in the image data; the prompt word is used to guide the part recognition model to analyze and infer the area occupied by the part in the image data; the image of the part in the image data is segmented according to the target bounding box, a plurality of candidate segmentation masks of the part and a confidence degree of each candidate segmentation mask are obtained; the hollowing rate of the part corresponding to each candidate segmentation mask is determined respectively to obtain a candidate hollowing rate set; and the target hollowing rate is determined from the candidate hollowing rate set according to the confidence degree. The method can simplify the operation process on the premise of accurately determining the part hollowing rate.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, device, medium, and program product for determining the hollowness rate of components. Background Technology

[0002] In the design, manufacturing, and cost assessment of components, the hollowness rate is a key parameter affecting the weight, structural strength, and manufacturing costs of components. For example, the hollowness rate of a wheel rim is a crucial parameter affecting its weight and wind resistance characteristics, and is of great significance in weight prediction and cost accounting. However, related technologies often require redesigning the hollowness rate detection process or re-collecting large amounts of data for model training for different styles and types of components, resulting in cumbersome operations and difficulty in guaranteeing accuracy. Summary of the Invention

[0003] Based on this, this application addresses the aforementioned technical problems by providing a method, apparatus, equipment, medium, and program product for determining the hollowness rate of components, which simplifies the operation process while accurately determining the hollowness rate of components.

[0004] Firstly, this application provides a method for determining the hollowness ratio of a component, including:

[0005] Image data and preset prompts are input into a pre-trained component recognition model to obtain the target bounding box output by the component recognition model; the target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data;

[0006] Based on the target bounding box, the images of the components in the image data are segmented to obtain multiple candidate segmentation masks for the components and the confidence level of each candidate segmentation mask;

[0007] The cutout ratio of the component corresponding to each candidate segmentation mask is determined to obtain the candidate cutout ratio set;

[0008] The target cutout rate is determined from the candidate cutout rate set based on the confidence level.

[0009] In the aforementioned method for determining the hollowness rate of components, a pre-trained component recognition model is used. Component detection is performed based on the component image data and preset prompts to obtain the target bounding box. This allows for component detection by simply adjusting the prompts for different components. Multiple candidate segmentation masks are generated from the component image and the target bounding box. The hollowness rate of the component corresponding to each candidate segmentation mask is determined, and the target hollowness rate is determined based on the confidence level of each candidate segmentation mask. Hollowness rate calculations can be performed separately for different candidate segmentation masks in different components. The target hollowness rate is determined by combining the hollowness rates corresponding to different candidate segmentation masks. This helps reduce errors caused by differences in the shape of the segmentation masks and ensures the accuracy of hollowness rate calculations for different components. Thus, the overall operation process is simplified while accurately determining the hollowness rate of different components.

[0010] In an optional embodiment of the first aspect, the component recognition model includes an open vocabulary detection model. Image data and preset prompts are input into the pre-trained component recognition model to obtain a target bounding box output by the component recognition model. This includes: inputting image data and preset prompts into the pre-trained open vocabulary detection model to obtain multiple candidate bounding boxes output by the open vocabulary detection model and the confidence level of each candidate bounding box; and determining the target bounding box from each candidate bounding box based on the confidence level of each candidate bounding box.

[0011] In this optional embodiment, the open vocabulary detection model is used to detect components based on image data and prompt words, and the target bounding box is determined according to the confidence of each of the multiple candidate bounding boxes. This can utilize the generalization ability of the open vocabulary detection model to achieve accurate positioning of components, thereby improving the accuracy of determining the hollowness rate of components.

[0012] In an optional embodiment of the first aspect, the constraints on the open vocabulary detection model include: a bounding box confidence threshold, used to constrain the range of confidence values ​​of the candidate bounding boxes output by the open vocabulary detection model.

[0013] In this optional embodiment, the range of confidence values ​​for candidate bounding boxes is constrained by a bounding box confidence threshold to effectively filter them during the component detection process of the open vocabulary detection model. This ensures the credibility of the candidate bounding boxes output by the open vocabulary detection model, which is beneficial for the accurate positioning of components and improves the accuracy of determining the hollowness rate of components.

[0014] In an optional embodiment of the first aspect, the constraints on the open vocabulary detection model include: a text matching degree threshold, used to constrain the range of values ​​for the text matching degree of the candidate bounding boxes output by the open vocabulary detection model, wherein the text matching degree is used to quantify the degree of association between the image region marked by the candidate bounding box and the prompt word.

[0015] In this optional embodiment, the range of text matching degree of candidate bounding boxes is constrained by the text matching degree threshold to effectively filter them in the component detection process of the open vocabulary detection model. This can ensure the credibility of the candidate bounding boxes output by the open vocabulary detection model, which is beneficial to the accurate positioning of components and improves the accuracy of determining the hollowing rate of components.

[0016] In an optional embodiment of the first aspect, the method for determining the hollowing rate of components further includes: generating a prompt message when the confidence scores of each candidate bounding box are all less than a preset first confidence threshold, so as to indicate that the image data does not support the determination of the hollowing rate.

[0017] In this optional embodiment, when the confidence scores of each candidate bounding box are all less than a preset first confidence threshold, a prompt message is sent to indicate that the image does not support the determination of the cutout rate. This can avoid determining the cutout rate based on unreliable detection results as much as possible, which helps to reduce the transmission of erroneous results and thus enhances the accuracy and reliability of the cutout rate determination.

[0018] In an optional embodiment of the first aspect, the image of a component in the image data is segmented according to the target bounding box to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask, including: extracting multi-scale features of the component from the image data; and using a pre-trained image segmentation model to segment the image of the component in the image data based on the multi-scale features and the target bounding box to obtain multiple candidate segmentation masks and the confidence level of each candidate segmentation mask.

[0019] In this optional embodiment, by extracting multi-scale features of image data, an image segmentation model is used in conjunction with the target bounding box for segmentation to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask. This allows for segmentation using multi-scale features of the image, ensuring the accuracy and precision of the segmentation, thereby improving the accuracy of the candidate segmentation masks.

[0020] In an optional embodiment of the first aspect, the cutout ratio of the component corresponding to each candidate segmentation mask is determined to obtain a candidate cutout ratio set, including: for each candidate segmentation mask, determining the envelope geometry based on the outer contour of the candidate segmentation mask; determining the intersection area between the solid region of the component in the candidate segmentation mask and the envelope geometry according to the candidate segmentation mask and the envelope geometry; determining the reference area of ​​the envelope geometry, and obtaining the cutout ratio of the component corresponding to the candidate segmentation mask according to the intersection area and the reference area; and obtaining a candidate cutout ratio set according to the cutout ratio corresponding to each candidate segmentation mask.

[0021] In this optional embodiment, the envelope geometry is determined based on the outer contour of the candidate segmentation mask. The intersection area between the entity region and the envelope geometry is determined by the candidate segmentation mask and the envelope geometry. The cutout rate is determined by combining the reference area of ​​the envelope geometry. A unified area reference can be established based on the reference area of ​​the envelope geometry, so that parts of various sizes and shapes can be calculated for cutout rate under the same measurement system, ensuring the accuracy of cutout rate calculation.

[0022] In an optional embodiment of the first aspect, determining the target cutout rate from the candidate cutout rate set based on the confidence level includes: determining the target segmentation mask from each candidate segmentation mask based on the confidence level; and obtaining the target cutout rate of the component based on the cutout rate of the target segmentation mask when the cutout rate corresponding to the target segmentation mask is within a preset cutout rate range.

[0023] In this optional embodiment, the target segmentation mask is determined based on the confidence level of each candidate segmentation mask, and the perforation rate of the target segmentation mask is checked for reasonableness based on a preset perforation rate range. When the reasonableness check is passed, the target perforation rate of the component is obtained based on the perforation rate of the target segmentation mask, thereby improving the anti-interference capability and accuracy of perforation rate detection.

[0024] In an optional embodiment of the first aspect, the method for determining the component cutout rate further includes: determining each remaining segmentation mask when the cutout rate of the target segmentation mask is not within the cutout rate range; the remaining segmentation masks include candidate segmentation masks other than the target segmentation mask among the candidate segmentation masks; determining intermediate segmentation masks from each remaining segmentation mask based on their respective confidence level and cutout rate; the confidence level of the intermediate segmentation mask exceeds a preset second confidence level threshold, and the cutout rate of the intermediate segmentation mask is within the cutout rate range; when the intermediate segmentation mask is not empty, obtaining the target cutout rate of the component based on the respective confidence level and cutout rate of each intermediate segmentation mask; when the intermediate segmentation mask is empty, obtaining the target cutout rate of the component based on the cutout rate of the target segmentation mask.

[0025] In this optional embodiment, when the cutout rate of the target segmentation mask fails the rationality check, an intermediate segmentation mask that simultaneously meets the confidence threshold and the cutout rate range can be selected from the remaining candidate masks to determine the target cutout rate of the component, which helps to improve the accuracy of cutout rate detection; moreover, when the intermediate segmentation mask is empty, the target segmentation mask is backed up to obtain the target cutout rate, ensuring the stability of the cutout rate detection process.

[0026] Secondly, this application also provides a device for determining the hollowness ratio of components, comprising:

[0027] The component detection module is used to input image data and preset prompts into a pre-trained component recognition model to obtain the target bounding box output by the component recognition model. The target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data.

[0028] The segmentation mask generation module is used to segment the image of the component in the image data according to the target bounding box, and obtain multiple candidate segmentation masks of the component and the confidence level of each candidate segmentation mask;

[0029] The cutout rate determination module is used to determine the cutout rate of each component corresponding to each candidate segmentation mask, and obtain a candidate cutout rate set;

[0030] The detection result acquisition module is used to determine the target cutout rate from the candidate cutout rate set based on the confidence level.

[0031] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0033] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above aspects.

[0034] Regarding the beneficial effects of any of the technical solutions in the second to fifth aspects mentioned above, refer to the beneficial effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of an optional application environment for a method for determining the hollowness rate of components in one embodiment.

[0037] Figure 2This is a schematic diagram of an optional process for determining the hollowness rate of components in one embodiment;

[0038] Figure 3 This is a schematic diagram of an optional process for calculating the hollowness ratio in one embodiment;

[0039] Figure 4 This is a schematic diagram of an optional process for determining the rim void ratio in one embodiment;

[0040] Figure 5 This is a schematic diagram of an optional process for detection and segmentation based on a wheel rim image in one embodiment;

[0041] Figure 6 This is a schematic diagram of an optional process for calculating the rim void ratio in one embodiment;

[0042] Figure 7 This is a schematic diagram of an optional flowchart for the backoff mechanism of the hollowing rate calculation in one embodiment;

[0043] Figure 8 This is a schematic diagram of an optional structure of a component hollowing rate determination device in one embodiment;

[0044] Figure 9 This is a schematic diagram of an optional internal structure of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0046] The terms "first," "second," etc., used in this application may be used to describe various elements, but these elements are not limited by these terms. These terms are used only to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0047] The method for determining the hollowness ratio of components provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 101 communicates with server 102 via a network, and a data storage system stores the data that server 102 needs to process. The data storage system can be integrated onto server 102 or located in the cloud or on another network server. Terminal 101 can capture images of components to obtain image data of the components and send this image data to server 102. Server 102 can input the image data and preset prompts into a pre-trained component recognition model for component detection. The component recognition model outputs target bounding boxes, and based on these bounding boxes, it segments the image of the components in the image data to obtain multiple candidate segmentation masks for the components and the confidence level of each candidate segmentation mask. For each candidate segmentation mask, server 102 can determine the corresponding component's cutout rate, and a candidate cutout rate set can be obtained based on the cutout rates of each candidate segmentation mask. Server 102 can determine the target cutout rate of the component from the candidate cutout rate set based on the confidence level of each candidate segmentation mask. In some embodiments, server 102 can send the target cutout rate of the component to terminal 101 for feedback to the user through terminal 101.

[0048] In some embodiments, the method for determining the hollowness ratio of a component can also be implemented separately by the terminal 101 or the server 102. For example, the terminal 101 can directly perform detection based on the captured image data of the component to obtain the target hollowness ratio of the component. Alternatively, the server 102 can obtain image data of the component from the data storage system and perform detection based on the obtained image data to obtain the target hollowness ratio of the component.

[0049] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0050] In one exemplary embodiment, such as Figure 2 As shown, a method for determining the hollowness ratio of components is provided, and this method is applied to... Figure 1 Taking computer devices such as terminals and / or servers as examples, the explanation includes the following steps S201 to S204. Wherein:

[0051] Step S201: Input the image data and preset prompts into the pre-trained component recognition model to obtain the target bounding box output by the component recognition model; the target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data.

[0052] Among them, components refer to physical objects for which the porosity needs to be determined. Components can include, but are not limited to, at least one of the following physical objects: vehicle wheel rims, ventilation grille panels and fan blades in industrial radiators, metal chair backs, speaker grilles, heat dissipation vent panels, and lamp shades in electronic device housings. Porosity (also known as open area ratio or porosity) refers to the proportion of the area of ​​holes per unit area, and can be used to balance the strength, weight, ventilation, heat dissipation, or aesthetic requirements of components.

[0053] Image data can be acquired for components with a defined hollowness ratio. For example, image data can include images of the components, which can be obtained by taking photos of the components, such as from a frontal view. Cue words are used to guide component detection, such as guiding the component recognition model to analyze and infer the area occupied by the component in the image data. Cue words can be pre-set based on the component, such as being constructed based on the component type, thus providing the component recognition model with the type of component to be detected. For example, if the component is a wheel rim, the pre-set clue words for that component could include the strings "wheel rim" or "wheel rim"; similarly, if the component is a grille panel, the pre-set clue words for that component could include the strings "ventilationgrille" or the component product number "GR-001".

[0054] The component recognition model can include pre-trained neural network models, such as, but not limited to, at least one of the following models: GLIP (Grounded Language-Image Pre-training), SiamRPN (Siamese Region Proposal Network), Grounding DINO (Grounding Detection Transformer with Improved deNoising anchors), OWL-ViT (Open-World Localization Vision Transformer), and CLIP (Contrastive Language-Image Pre-training). The component recognition model can support multimodal input, receiving image data and prompts. Guided by the prompts, it analyzes and infers the region occupied by the component in the image data and outputs the corresponding target bounding box.

[0055] The target bounding box is used to define the area occupied by a component in the image data of the component. The target bounding box can be a rectangle, which can be represented by the coordinates of the center point and the width and height (cx, cy, w, h) of the rectangle, or by the coordinates of the top left and bottom right corners (x1, y1, x2, y2). Here, cx is the x-coordinate of the center point of the rectangle, cy is the y-coordinate of the center point, w is the width of the rectangle, and h is the height of the rectangle; x1 and y1 can be the x and y coordinates of the top left corner of the rectangle, and x2 and y2 can be the x and y coordinates of the bottom right corner. For example, if the image of the component is a 1024×1024 pixel photograph of a lampshade, with the lampshade located in the center, the target bounding box obtained after detection can include a rectangular area with center coordinates (512, 512) and width and height (800, 800), indicating that the lampshade is located within the rectangular area corresponding to this target bounding box.

[0056] For example, a computer device can acquire image data containing components. This image data can be acquired by capturing images of the components, such as by taking pictures of the components from a frontal viewpoint. The computer device can acquire preset prompts for the components. These prompts can be constructed based on the type of the component. The type of the component can be preset by the user or obtained by the computer device through target recognition of the component image data. For example, the computer device can call a pre-trained image classification model, such as ResNet (Residual Network) or ViT (Vision Transformer), to perform target recognition on the images of components in the image data to determine the type of components included in the image. The image classification model can be trained using supervised learning based on image samples including various components. Each image sample can carry a corresponding true type label, which can be pre-annotated manually.

[0057] The computer device can pre-build prompt words based on the type of the component. For example, if the component type is "wheel rim," the corresponding prompt word could include "wheel rim"; if the component type is "grille panel," the corresponding prompt word could include "radiator grille." In some embodiments, the computer device can build the corresponding prompt words in real time after determining the component type, or it can pre-build candidate prompt words for various types of components and determine the required prompt word from the candidate prompt words of the corresponding type after determining the component type. For example, the computer device can pre-build corresponding candidate prompt words for various types of components, and one or more candidate prompt words can be built for each type of component. After determining the type of the current component, the computer device can query the corresponding pre-built candidate prompt words according to the type of the current component and determine the prompt word required for this component inspection from the candidate prompt words.

[0058] Computer devices can call a pre-trained component recognition model and input image data and preset prompts into the component recognition model to detect components. Guided by the prompts, the component recognition model can analyze and infer the area occupied by the component in the image data and output the target bounding box used to mark the area occupied by the component in the image data.

[0059] Step S202: Based on the target bounding box, segment the image of the component in the image data to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask.

[0060] The candidate segmentation mask can be the segmentation result obtained by segmenting the solid regions that belong to the component and other regions that do not belong to the component in the image of the component. The candidate segmentation mask can identify the solid regions that belong to the component and the other regions that do not belong to the component. For example, in the candidate segmentation mask, a pixel with a value of 1 belongs to the solid region of the component, and a pixel with a value of 0 belongs to other regions. Other regions can include the background region of the component and the hollowed-out region of the component. There can be multiple candidate segmentation masks, such as two or more, and each candidate segmentation mask represents a segmentation hypothesis for the solid regions and other regions of the component.

[0061] The confidence score of a candidate segmentation mask characterizes its reliability. The confidence score can range from 0 to 1, with a higher score indicating a more reliable correspondence between the candidate segmentation mask and the actual contour of the component. The confidence score of a candidate segmentation mask can be obtained when segmenting the component image based on the target bounding box. That is, while segmenting the component image in the image data based on the target bounding box, in addition to obtaining multiple candidate segmentation masks for the component, the confidence score of each candidate segmentation mask can also be obtained.

[0062] Optionally, the computer device can segment the image of a component in the image data based on the target bounding box, obtaining multiple candidate segmentation masks for the component and the confidence score of each candidate segmentation mask. For example, the computer device can call a preset image segmentation model, such as at least one of various models such as Mask R-CNN (Mask Region-based Convolutional Neural Network), SAM2 (Segment Anything Model 2), EfficientSAM (Efficient Segment Anything Model), MobileSAM (Mobile Segment Anything Model), or FastSAM (Fast Segment Anything Model). The image segmentation model can receive the image and one or more spatial cue information as input, and output the candidate segmentation mask corresponding to the image and the confidence score of each candidate segmentation mask. Computer equipment can input images of components and target bounding boxes from image data into an image segmentation model. The image segmentation model then segments the image to identify the entity regions belonging to the components and other regions not belonging to the components within the target bounding box. It outputs multiple candidate segmentation masks for the components and the confidence level of each candidate mask. In some embodiments, the image segmentation model can support multiple mask outputs; that is, instead of outputting a single deterministic result, it outputs multiple candidate segmentation masks of different forms and the confidence level of each candidate mask under the same target bounding box prompt. Each candidate segmentation mask represents a segmentation hypothesis of the image segmentation model for the image within the target bounding box, and the confidence level of the candidate segmentation mask represents the reliability of the segmentation hypothesis as determined by the image segmentation model.

[0063] Step S203: Determine the cutout ratio of the component corresponding to each candidate segmentation mask to obtain the candidate cutout ratio set.

[0064] The candidate cutout rate set can include cutout rates determined based on each candidate segmentation mask. The cutout rate can be calculated independently for each candidate segmentation mask, and the cutout rate can range from 0% to 100%.

[0065] For example, the computer device can determine the corresponding cutout ratio for each of the generated multiple candidate segmentation masks, and obtain a candidate cutout ratio set based on the cutout ratio of each candidate segmentation mask. For each candidate segmentation mask, the computer device can extract the outer contour of the candidate segmentation mask, such as by using the findContours interface of OpenCV (OpenSource Computer Vision Library) to extract the contour of the candidate segmentation mask. The computer device can determine the envelope geometry based on the outer contour, such as by fitting the outer contour based on the minimum circumcircle fitting function, the minimum circumcircle ellipse fitting function, or the convex hull fitting function to determine the envelope geometry, thereby obtaining the geometric parameters of the envelope geometry, such as the center coordinates and radius of the minimum circumcircle, the center, major and minor axes and angles of the minimum circumcircle ellipse, or the coordinates of each vertex of the convex hull. The computer device can calculate the cutout ratio of the candidate segmentation mask based on the candidate segmentation mask and the envelope geometry. In some embodiments, the computer device can determine a reference area for the envelope geometry, such as obtaining the reference area based on the geometric parameters of the envelope geometry, and obtaining the cutout ratio corresponding to the candidate segmentation mask based on the candidate segmentation mask and the reference area. After traversing each candidate segmentation mask, the computer device can obtain the cutout ratio calculated independently for each candidate segmentation mask, and obtain a set of candidate cutout ratios based on the cutout ratios of each candidate segmentation mask.

[0066] Step S204: Determine the target cutout rate from the candidate cutout rate set based on the confidence level.

[0067] The target cutout rate is the detection result obtained by cutting out the component, such as 5%, 12%, or 40%. Optionally, the computer device can determine the target cutout rate of the component from the candidate cutout rate set by combining the confidence levels of each candidate segmentation mask. For example, when the cutout rate corresponding to the candidate segmentation mask with the highest confidence value is within a preset cutout rate range, the computer device can obtain the target cutout rate of the component based on the cutout rate corresponding to that candidate segmentation mask. The cutout rate range can be a pre-defined reasonable value range for the cutout rate, such as a range of 3.5% to 47.5%.

[0068] In the aforementioned method for determining the hollowness rate of components, a pre-trained component recognition model is used. Component detection is performed based on the component image data and preset prompts to obtain the target bounding box. This allows for component detection by simply adjusting the prompts for different components. Multiple candidate segmentation masks are generated from the component image and the target bounding box. The hollowness rate of the component corresponding to each candidate segmentation mask is determined, and the target hollowness rate is determined based on the confidence level of each candidate segmentation mask. Hollowness rate calculations can be performed separately for different candidate segmentation masks in different components. The target hollowness rate is determined by combining the hollowness rates corresponding to different candidate segmentation masks. This helps reduce errors caused by differences in the shape of the segmentation masks and ensures the accuracy of hollowness rate calculations for different components. Thus, the overall operation process is simplified while accurately determining the hollowness rate of different components.

[0069] In an exemplary embodiment, the component recognition model includes an open vocabulary detection model. Image data and preset prompts are input into the pre-trained component recognition model to obtain the target bounding box output by the component recognition model. This includes: inputting image data and preset prompts into the pre-trained open vocabulary detection model to obtain multiple candidate bounding boxes and the confidence score of each candidate bounding box; and determining the target bounding box from each candidate bounding box based on the confidence score of each candidate bounding box.

[0070] The component recognition model can include an open vocabulary detection model. This model can locate and identify target objects in an image based on arbitrary text descriptions (rather than a pre-defined fixed list of categories), thus detecting target objects not learned during training. For example, an open vocabulary detection model can include, but is not limited to, at least one of GLIP, Grounding DINO, OWL-ViT, and CLIP models. Component detection using an open vocabulary detection model eliminates the need for pre-fixed identifiers for various component types and for fine-tuning the model using local data. It can locate the area occupied by a component in an image based on prompts, making it suitable for scenarios with only image input and a lack of dedicated detection datasets.

[0071] Candidate bounding boxes are initially identified by the open vocabulary detection model in an image and are used to define the regions occupied by components within the image. For example, candidate bounding boxes can be rectangles, which can be represented by the coordinates of their center point and width / height (cx, cy, w, h), or by the coordinates of their top-left and bottom-right corners (x1, y1, x2, y2). The confidence score of a candidate bounding box represents the degree of certainty the open vocabulary detection model has regarding whether the obtained candidate bounding box actually contains a component. Each candidate bounding box can have a corresponding confidence score. The confidence score of a candidate bounding box can be a value from 0 to 1, and a higher confidence score indicates a higher degree of certainty from the open vocabulary detection model that the candidate bounding box contains a component.

[0072] For example, a computer device can invoke a pre-trained open vocabulary detection model and input image data and preset prompts into the open vocabulary detection model. The open vocabulary detection model can then detect components based on the image and the prompts. Guided by the prompts, the open vocabulary detection model can analyze and infer the area occupied by the component in the image data and output multiple candidate bounding boxes corresponding to the component in the image, along with the confidence score of each candidate bounding box. In some embodiments, the open vocabulary detection model can match each image region in the component image with the prompts, determine the candidate bounding boxes corresponding to the component in the image based on the matching scores, and output multiple candidate bounding boxes and their respective confidence scores. The computer device can then combine the confidence scores of each candidate bounding box to determine the target bounding box. For example, the computer device can determine the candidate bounding box with the highest confidence score as the target bounding box. For instance, if the open vocabulary detection model outputs two candidate bounding boxes, with candidate bounding box 1 having a confidence score of 0.91 and candidate bounding box 2 having a confidence score of 0.67, the computer device can determine candidate bounding box 1 as the target bounding box.

[0073] In this exemplary embodiment, the open vocabulary detection model is used to detect components based on image data and prompt words, and the target bounding box is determined according to the confidence of each of the multiple candidate bounding boxes. The generalization ability of the open vocabulary detection model can be used to achieve accurate positioning of components, thereby improving the accuracy of determining the hollowness rate of components.

[0074] In an exemplary embodiment, the constraints on the open vocabulary detection model include: a bounding box confidence threshold, which is used to constrain the range of confidence values ​​of the candidate bounding boxes output by the open vocabulary detection model.

[0075] The bounding box confidence threshold is used to constrain the range of confidence values ​​for candidate bounding boxes output by the open vocabulary detection model. By setting this threshold, the confidence of candidate bounding boxes output by the open vocabulary detection model can be kept within the desired range. For example, the bounding box confidence threshold can constrain the open vocabulary detection model to output candidate bounding boxes with confidence values ​​exceeding the threshold, ensuring that the confidence of each candidate bounding box output by the open vocabulary detection model exceeds the threshold. The bounding box confidence threshold can be preset according to the application scenario of component detection and the desired detection accuracy and recall rate. For example, in the rim detection scenario, the bounding box confidence threshold can be preset to 0.4, thus constraining the open vocabulary detection model to only output candidate bounding boxes with confidence values ​​exceeding 0.4.

[0076] Optionally, constraints can be imposed on the open vocabulary detection model to control its output. For example, a bounding box confidence threshold can be set to constrain the range of confidence values ​​for the candidate bounding boxes output by the open vocabulary detection model. In some embodiments, the open vocabulary detection model can perform component detection based on an image under the guidance of prompts, generating bounding boxes corresponding to the components and their confidence values. The open vocabulary detection model can then judge the generated bounding boxes according to the bounding box confidence threshold to determine whether they need to be output. For example, bounding boxes with confidence values ​​exceeding the threshold can be output as candidate bounding boxes; bounding boxes with confidence values ​​not exceeding the threshold can be discarded. This allows the open vocabulary detection model to output candidate bounding boxes with confidence values ​​exceeding the threshold and their corresponding confidence values.

[0077] In this exemplary embodiment, the confidence threshold of the bounding box is used to constrain the range of confidence values ​​of the candidate bounding box, so as to effectively screen them in the component detection process of the open vocabulary detection model. This can ensure the credibility of the candidate bounding boxes output by the open vocabulary detection model, which is beneficial to the accurate positioning of components and improves the accuracy of determining the hollowness rate of components.

[0078] In an exemplary embodiment, the constraints on the open vocabulary detection model include: a text matching degree threshold, which is used to constrain the range of text matching degree values ​​of the candidate bounding boxes output by the open vocabulary detection model. The text matching degree is used to quantify the degree of association between the image region marked by the candidate bounding box and the prompt word.

[0079] The text matching threshold is used to constrain the range of text matching scores for candidate bounding boxes output by the open vocabulary detection model. By setting this threshold, the text matching scores of candidate bounding boxes can be kept within the desired range. Text matching score quantifies the correlation between the image region marked by the candidate bounding box and the prompt word. A higher text matching score indicates a stronger correlation between the image region and the prompt word, meaning the image region marked by the candidate bounding box better meets the detection requirements of the prompt word and is more likely to belong to the area occupied by the component. For example, the text matching threshold can constrain the open vocabulary detection model to output candidate bounding boxes with text matching scores exceeding the threshold, ensuring that each candidate bounding box output by the open vocabulary detection model has a text matching score exceeding the threshold. The text matching threshold can be preset according to the application scenario of component detection and the desired detection accuracy and recall. For example, in the rim detection scenario, the text matching threshold can be preset to 0.3, thus constraining the open vocabulary detection model to only output candidate bounding boxes with text matching scores exceeding 0.3.

[0080] Optionally, constraints can be imposed on the open vocabulary detection model to control its output. For example, a text matching threshold constraint can be set to limit the range of the text matching threshold values ​​for the candidate bounding boxes output by the open vocabulary detection model. In some embodiments, the open vocabulary detection model can perform component detection based on images under the guidance of prompt words, generate bounding boxes corresponding to the components, and determine the degree of association between the image regions marked by the bounding boxes and the prompt words, thereby obtaining the text matching degree of the bounding boxes. The open vocabulary detection model can judge the generated bounding boxes according to the text matching threshold to determine whether they need to be output. For example, bounding boxes with a text matching degree exceeding the text matching threshold can be output as candidate bounding boxes; bounding boxes with a text matching degree not exceeding the text matching threshold can be discarded by the open vocabulary detection model, thereby enabling the open vocabulary detection model to output candidate bounding boxes with a text matching degree exceeding the text matching threshold.

[0081] In some embodiments, constraints such as a bounding box confidence threshold and a text matching threshold can be simultaneously applied to the open vocabulary detection model, thereby enabling the open vocabulary detection model to output candidate bounding boxes with confidence exceeding the bounding box confidence threshold and text matching exceeding the text matching threshold.

[0082] In this exemplary embodiment, the range of text matching degree values ​​for candidate bounding boxes is constrained by a text matching degree threshold to effectively filter them during the component detection process of the open vocabulary detection model. This ensures the credibility of the candidate bounding boxes output by the open vocabulary detection model, which is beneficial for the accurate positioning of components and improves the accuracy of determining the hollowing rate of components.

[0083] In an exemplary embodiment, the method for determining the hollowing rate of components further includes: generating a prompt message when the confidence scores of each candidate bounding box are all less than a preset first confidence threshold, so as to indicate that the image data does not support the determination of the hollowing rate.

[0084] The first confidence threshold is used to evaluate the overall reliability of all candidate bounding boxes output by the open vocabulary detection model for the current component image. The value of the first confidence threshold can be adjusted according to different component types, image acquisition conditions, or accuracy requirements; for example, the first confidence threshold can include a pre-set 0.55 or 0.6. If the confidence of each candidate bounding box does not exceed the pre-set first confidence threshold, it can be determined that there is no reliable component region in the image, i.e., the overall reliability of the image is low, and anomaly processing can be triggered. For example, a prompt message can be generated to indicate that the current image does not support the determination of the cutout rate and that the component image needs to be replaced for cutout rate determination. The form of the prompt message can be flexibly configured according to actual needs; for example, the prompt message can include various forms of information such as text, images, audio, and video.

[0085] For example, the computer device can compare the confidence scores of each candidate bounding box with a preset first confidence threshold. If the confidence score of each candidate bounding box is less than the first confidence threshold, it can be determined that there is no reliable component region in the image. The computer device can then generate a prompt message and output it to the user interface to indicate that the image does not support the determination of the cutout rate. For instance, if an open vocabulary detection model outputs three candidate bounding boxes with confidence scores of 0.35, 0.32, and 0.22 respectively, and the first confidence threshold is 0.55, the computer device can generate a prompt message indicating that no reliable component region was detected in the current component image, and therefore cutout rate determination is not supported. The message requests that the image be replaced or the detection conditions adjusted for re-detection. When at least one candidate bounding box has a confidence score exceeding the first confidence threshold, the computer device can determine the target bounding box from among the candidate bounding boxes.

[0086] In this exemplary embodiment, when the confidence scores of each candidate bounding box are all less than a preset first confidence threshold, a prompt message is sent to indicate that the image does not support the determination of the cutout rate. This can avoid continuing to determine the cutout rate based on unreliable detection results as much as possible, which helps to reduce the transmission of erroneous results and thus enhances the accuracy and reliability of the cutout rate determination.

[0087] In an exemplary embodiment, the image of a component in the image data is segmented based on the target bounding box to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask. This includes: extracting multi-scale features of the component from the image data; and using a pre-trained image segmentation model to segment the image of the component in the image data based on the multi-scale features and the target bounding box to obtain multiple candidate segmentation masks and the confidence level of each candidate segmentation mask.

[0088] The multi-scale features can be obtained by extracting features from the image data of the parts at multiple scales, such as original size, half size, quarter size, and eighth size. The image segmentation model can be a pre-trained deep learning neural network model that receives an image and one or more spatial cues as input and outputs a candidate segmentation mask corresponding to the image. For example, the image segmentation model can include, but is not limited to, at least one of various models such as Mask R-CNN, SAM2, EfficientSAM, MobileSAM, or FastSAM.

[0089] Optionally, the computer device can extract multi-scale features of the components from the image data of the components. For example, the computer device can perform a series of convolutions, pooling, self-attention, and other processes on the image data of the components through a pre-trained feature extraction network to extract features from the image data of the components and obtain multi-scale features. The computer device can call a pre-trained image segmentation model and input the multi-scale features and target bounding boxes into the image segmentation model so that the image segmentation model can segment the image and output multiple candidate segmentation masks for the components and the confidence score of each candidate segmentation mask. In some embodiments, the image segmentation model may include a SAM2 model. The computer device can call the image encoder of the SAM2 model and extract multi-scale features from the image of the components through the image encoder; the SAM2 model can segment the image based on the multi-scale features and output multiple candidate segmentation masks for the components and the confidence score of each candidate segmentation mask.

[0090] In this exemplary embodiment, by extracting multi-scale features from image data, an image segmentation model is used in conjunction with the target bounding box for segmentation to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask. This allows for segmentation using multi-scale features of the image, ensuring the accuracy and precision of the segmentation, thereby improving the accuracy of the candidate segmentation masks.

[0091] In one exemplary embodiment, such as Figure 3 As shown, the process of calculating the cutout rate involves determining the cutout rate of the component corresponding to each candidate segmentation mask to obtain a candidate cutout rate set, including steps S301 to S304. Wherein:

[0092] Step S301: For each candidate segmentation mask, determine the envelope geometry based on the outer contour of the candidate segmentation mask.

[0093] The outer contour is used to characterize the shape boundary of the component in the image. The outer contour can include an array of coordinates of a series of consecutive pixels. In some embodiments, the outer contour can be extracted from the candidate segmentation mask based on an edge detection algorithm. For example, the `findContours` function can be called, and the contour retrieval mode can be set to `RETR_EXTERNAL`, thereby obtaining only the outermost boundary from the candidate segmentation mask to obtain the outer contour of the candidate segmentation mask. The envelope geometry is a geometric shape that fits the outer contour and can enclose the overall external shape of the component. The envelope geometry can include the minimum circumcircle (calculated using the `minEnclosingCircle` function), the minimum circumcircle ellipse (calculated using the `fitEllipse` function), or the convex hull (calculated using the `convexHull` function). The envelope geometry can provide a unified reference area for calculating the hollowness ratio, allowing components of different shapes to be compared and calculated according to a unified geometric scale.

[0094] For example, a computer device can determine the corresponding envelope geometry for each of the generated multiple candidate segmentation masks. For each candidate segmentation mask, the computer device can extract its outer contour, such as by using the findContours interface of OpenCV (Open Source Computer Vision Library) to extract the contour of the candidate segmentation mask. The computer device can then determine the envelope geometry based on this outer contour, such as by fitting the outer contour to a minimum circumcircle fitting function, a minimum circumcircle ellipse fitting function, or a convex hull fitting function, thereby obtaining the geometric parameters of the envelope geometry, such as the center coordinates and radius of the minimum circumcircle, the center, major and minor axes, and angles of the minimum circumcircle ellipse, or the coordinates of each vertex of the convex hull.

[0095] Step S302: Based on the target candidate segmentation mask and envelope geometry, determine the intersection area between the solid region of the component in the target candidate segmentation mask and the envelope geometry.

[0096] In this segmentation mask, the entity region can be an image region belonging to the components of a part. The candidate segmentation mask can include entity regions and other regions. The entity region is the image region belonging to the part, while the other regions are image regions in the candidate segmentation mask that do not belong to the part, such as the background region and / or hollowed-out region of the part. For example, for a candidate segmentation mask of a wheel rim, the entity region is the image region in the candidate segmentation mask representing the spokes, rim, and central entity of the wheel rim. Each pixel in this image region can be regarded as a solid component of the wheel rim. The intersection area characterizes the degree of overlap between the entity region and the envelope geometry in image space. The intersection area can be calculated based on the intersection area between the entity region and the envelope geometry, such as the number of pixels included in the intersection area between the entity region and the envelope geometry.

[0097] Optionally, the computer device can perform intersection calculations based on the candidate segmentation mask and the envelope geometry to obtain the intersection area between the solid region of the component and the envelope geometry in the candidate segmentation mask. In some embodiments, the computer device can generate a reference mask with the same resolution as the image of the component based on the envelope geometry, and perform a pixel-by-pixel comparison between the reference mask and the candidate segmentation mask to determine the pixels that are simultaneously valid in both the reference mask and the candidate segmentation mask, and obtain the intersection area between the solid region of the component and the envelope geometry in the candidate segmentation mask based on the number of each valid pixel.

[0098] Step S303: Determine the reference area of ​​the envelope geometry, and obtain the cutout rate of the component corresponding to the candidate segmentation mask based on the intersection area and the reference area.

[0099] The reference area can be the total area covered by the envelope geometry. The reference area can be calculated based on the geometric parameters of the envelope geometry. For example, if the envelope geometry is a minimum circumcircle, the computer device can calculate the reference area based on the radius of the minimum circumcircle; if the envelope geometry is a minimum circumcircle ellipse, the computer device can calculate the reference area based on the values ​​of the major and minor semi-axes of the minimum circumcircle ellipse; and if the envelope geometry is a convex hull, the computer device can calculate the reference area based on the coordinates of each vertex of the convex hull. In some embodiments, the computer device can also obtain the reference area based on the number of pixels included in the area covered by the envelope geometry.

[0100] Optionally, the computer device can determine the reference area of ​​the envelope geometry, such as by determining the reference area based on the geometric parameters of the envelope geometry. The computer device can calculate the cutout ratio corresponding to the candidate segmentation mask based on the intersection area and the reference area, such as by calculating the cutout ratio corresponding to the candidate segmentation mask as (reference area - intersection area) / reference area × 100%.

[0101] Step S304: Based on the cutout rate corresponding to each candidate segmentation mask, obtain the candidate cutout rate set.

[0102] For example, after traversing each candidate segmentation mask, the computer device can obtain the cutout ratio corresponding to each candidate segmentation mask, and obtain a set of candidate cutout ratios based on the cutout ratio corresponding to each candidate segmentation mask.

[0103] In this exemplary embodiment, the envelope geometry is determined based on the outer contour of the candidate segmentation mask. The intersection area between the entity region and the envelope geometry is determined by the candidate segmentation mask and the envelope geometry. The cutout rate is determined by combining the reference area of ​​the envelope geometry. A unified area reference can be established based on the reference area of ​​the envelope geometry, so that parts of various sizes and shapes can be calculated for cutout rate under the same measurement system, ensuring the accuracy of cutout rate calculation.

[0104] In an exemplary embodiment, determining the target cutout rate from the candidate cutout rate set based on confidence level includes: determining the target segmentation mask from each candidate segmentation mask based on confidence level; and obtaining the target cutout rate of the component based on the cutout rate of the target segmentation mask when the cutout rate corresponding to the target segmentation mask is within a preset cutout rate range.

[0105] The target segmentation mask is a selected candidate segmentation mask used for rationality verification. For example, the target segmentation mask can be the candidate segmentation mask with the highest confidence among all candidate segmentation masks. The hollowing-out rate range can be a pre-defined reasonable value range for the hollowing-out rate, used to characterize the physical constraints in the design of specific types of parts. For example, the hollowing-out rate range for wheel rims can be 3.5% to 47.5%, meaning the reasonable value of the hollowing-out rate should be no less than 3.5% and no more than 47.5%.

[0106] Optionally, the computer device can filter from the candidate segmentation masks based on their respective confidence scores to determine the target segmentation mask. For example, the candidate segmentation mask with the highest confidence score can be determined as the target segmentation mask. This target segmentation mask represents the most accurate segmentation result obtained when segmenting based on the image and the target bounding box. The computer device can determine a preset cutout rate range, which can be used to verify the reasonableness of the cutout rate. For example, the computer device can determine whether the cutout rate of the target segmentation mask is within the cutout rate range. If the cutout rate of the target segmentation mask is within the cutout rate range, it indicates that the cutout rate of the target segmentation mask has passed the reasonableness verification. The computer device can obtain the target cutout rate of the component based on the cutout rate of the target segmentation mask. For example, the computer device can use the cutout rate of the target segmentation mask as the target cutout rate of the component.

[0107] In some embodiments, when the cutout rate of the target segmentation mask is not within a preset cutout rate range, the computer device can obtain the target cutout rate of the component based on the confidence level and cutout rate of other candidate segmentation masks besides the target segmentation mask.

[0108] In this exemplary embodiment, the target segmentation mask is determined based on the confidence level of each candidate segmentation mask, and the perforation rate of the target segmentation mask is validated based on a preset perforation rate range. When the perforation rate is validated, the target perforation rate of the component is obtained based on the perforation rate of the target segmentation mask, thereby improving the anti-interference capability and accuracy of perforation rate detection.

[0109] In an exemplary embodiment, the component cutout rate determination method further includes: determining each remaining segmentation mask when the cutout rate of the target segmentation mask is not within the cutout rate range; the remaining segmentation masks include candidate segmentation masks other than the target segmentation mask among the candidate segmentation masks; determining intermediate segmentation masks from each remaining segmentation mask based on their respective confidence level and cutout rate; the intermediate segmentation mask has a confidence level exceeding a preset second confidence level threshold and its cutout rate is within the cutout rate range; when the intermediate segmentation mask is not empty, obtaining the target cutout rate of the component based on the respective confidence level and cutout rate of each intermediate segmentation mask; when the intermediate segmentation mask is empty, obtaining the target cutout rate of the component based on the cutout rate of the target segmentation mask.

[0110] The intermediate segmentation mask is a candidate segmentation mask selected from the remaining segmentation masks excluding the target segmentation mask to determine the target cutout rate. The intermediate segmentation mask must have a confidence level exceeding a preset second confidence threshold, and its cutout rate must be within the cutout rate range. The second confidence threshold can be set according to actual needs to filter the remaining segmentation masks; for example, it can be 0.4 or 0.55.

[0111] For example, when the cutout rate of the target segmentation mask is not within the cutout rate range, it indicates that the cutout rate of the target segmentation mask has not passed the reasonableness check. The computer device can then obtain all remaining segmentation masks except the target segmentation mask. The computer device can filter the remaining segmentation masks based on their respective confidence levels and cutout rates to determine the intermediate segmentation masks whose confidence levels exceed a preset second confidence threshold and whose cutout rates are within the cutout rate range. In other words, the intermediate segmentation masks ensure credibility at the segmentation level and pass the reasonableness check at the cutout rate level.

[0112] The computer device can determine whether the intermediate segmentation mask is empty. If the intermediate segmentation mask is empty, it indicates that among the remaining segmentation masks, there is no candidate segmentation mask that simultaneously guarantees credibility at the segmentation level and passes the reasonableness check at the cutout rate level. The computer device can obtain the target cutout rate of the component based on the cutout rate of the target segmentation mask. In some embodiments, a low-confidence flag can also be generated for the target cutout rate to prompt the user that the result may be inaccurate.

[0113] When the intermediate segmentation mask is not empty, it indicates that among the remaining segmentation masks, there exists at least one candidate segmentation mask that simultaneously guarantees confidence at the segmentation level and passes the rationality check at the cutout rate level. The computer device can then obtain the target cutout rate of the component based on the confidence level and cutout rate of each intermediate segmentation mask. For example, if the number of intermediate segmentation masks is 1, the computer device can directly obtain the target cutout rate of the component based on the cutout rate of that intermediate segmentation mask. Similarly, if the number of intermediate segmentation masks is greater than or equal to 2, the computer device can obtain the target cutout rate of the component based on the cutout rate corresponding to the intermediate segmentation mask with the highest confidence level.

[0114] In some embodiments, in addition to obtaining the target hollowing rate of the component, the computer device can also obtain the geometric parameters of the envelope geometry of the corresponding candidate segmentation mask, such as the center and radius of the minimum circumcircle, the confidence level of the candidate segmentation mask, and a visual illustration of the component (intuitively showing the hollowing situation of the component), thereby generating a hollowing rate detection result for the component.

[0115] In this exemplary embodiment, when the cutout rate of the target segmentation mask fails the rationality check, an intermediate segmentation mask that simultaneously satisfies the confidence threshold and the cutout rate range can be selected from the remaining candidate masks to determine the target cutout rate of the component, which helps to improve the accuracy of cutout rate detection. Moreover, when the intermediate segmentation mask is empty, the target segmentation mask is backed up to obtain the target cutout rate, ensuring the stability of the cutout rate detection process.

[0116] This application also provides an application scenario in which the above-described method for determining the hollowness ratio of components is applied. Specifically, the application of the method for determining the hollowness ratio of components in this scenario is as follows:

[0117] The components in this application scenario can be vehicle wheel rims, such as aluminum alloy wheels. The rim openness ratio (also known as the rim chamfer ratio) refers to the proportion of the area of ​​the open portion (spoke gaps) in the visible area of ​​the rim to the area of ​​the rim's outer circumcircle. The rim openness ratio is a key parameter affecting rim weight and wind resistance characteristics, and is of great significance in weight prediction and cost accounting.

[0118] In existing technologies for determining the hollowness ratio of wheel rims, manually annotating the hollow areas cannot support batch processing and does not meet the needs of high-throughput pricing scenarios. Segmentation methods based on color thresholds or fixed edge detection have poor adaptability to different lighting, backgrounds, and wheel rim colors (silver, black, matte, etc.) and insufficient generalization ability. Specialized training models require a large number of manually annotated segmentation masks, resulting in high data collection and annotation costs and poor transferability to new wheel rim styles (new spoke designs). Moreover, related technologies may generate masks that include the entire wheel rim (including spoke gap filling), making the hollowness ratio close to 0. The lack of anomaly detection and automatic backtracking error correction mechanisms makes the operation of determining the hollowness ratio of wheel rims cumbersome and difficult to guarantee accuracy. Based on this, the component hollowness ratio determination method provided in this application, when applied to the scenario of determining the hollowness ratio of wheel rims, can locate the wheel rim area, then perform image segmentation on the located target bounding box, calculate the hollowness ratio based on the envelope geometry, and improve the stability of the results through a rationality verification mechanism for candidate segmentation masks.

[0119] In this application scenario, a combination of Grounding DINO (as an open-vocabulary detection model) and SAM2 (as an image segmentation model) will be used as an example for explanation. Grounding DINO is an open-vocabulary detection model that combines the DINO visual base model and a language model, supporting the detection of targets of any category through natural language text prompts without requiring a category list; SAM2 is a general-purpose image segmentation model that supports various prompting methods such as points, bounding boxes, and masks. Figure 4 As shown, the methods for determining the rim openwork ratio include:

[0120] Step S401: Determine the image of the wheel rim, the cue word, the bounding box confidence threshold, and the text matching threshold. The cue word may include the text "wheel rim," the bounding box confidence threshold (box_threshold) can be 0.35, and the text matching threshold (text_threshold) can be 0.25.

[0121] Step S402: Rim detection is performed using an open vocabulary detection model. The open vocabulary detection model may include the Grounding DINO model.

[0122] Step S403 yields candidate bounding boxes and their confidence scores. The open vocabulary detection model can output multiple candidate bounding boxes and their respective confidence scores.

[0123] Step S404: Determine the target bounding box with the highest confidence level. The target bounding box can be represented by the center point coordinates and its width and height (cx, cy, w, h).

[0124] Step S405: Based on the target bounding box, segment the image of the wheel rim using an image segmentation model. The image segmentation model may include the SAM2 model.

[0125] Step S406 yields three candidate segmentation masks plus confidence scores.

[0126] Step S407: Determine the rim cutout ratio of each candidate segmentation mask. The rim cutout ratio can be calculated separately for each candidate segmentation mask to obtain the individual rim cutout ratio of each candidate segmentation mask. The outer contour of the candidate segmentation mask can be determined, and the minimum circumcircle of the outer contour can be determined, and a reference mask corresponding to the minimum circumcircle can be generated; based on the calculated rim cutout ratio of the candidate segmentation mask, the rim cutout ratio is obtained.

[0127] Step S408: Determine the target segmentation mask. The candidate segmentation mask with the highest confidence level can be determined as the target segmentation mask.

[0128] Step S409: Determine whether the rim cutout rate of the target segmentation mask is within the cutout rate range. If the rim cutout rate of the target segmentation mask is within the cutout rate range, proceed to step S410; if the rim cutout rate of the target segmentation mask is not within the cutout rate range, proceed to step S411.

[0129] Step S410: Obtain the hollowing rate detection result based on the rim hollowing rate of the target segmentation mask. The hollowing rate detection result may include the target hollowing rate determined for the rim.

[0130] Step S411: Obtain the hollowing rate detection result based on the hollowing rate of the rim using the remaining segmentation mask. For example, intermediate segmentation masks with a confidence level exceeding 0.5 and a hollowing rate within the hollowing rate range can be selected from the remaining segmentation masks, and the target hollowing rate can be determined based on the intermediate segmentation masks. The hollowing rate detection result may include, but is not limited to, the target hollowing rate of the rim, the center and radius of the smallest circumcircle, the confidence level, and a visual representation.

[0131] The process for determining the rim openwork ratio can be expressed as follows: ,in, To determine the final openwork ratio for the wheel rim; This describes the process of determining the perforation rate; Image of the wheel rim; The inputs for Grounding DINO include the cue word "wheel rim", the bounding box confidence threshold "box_threshold" = 0.35, and the text matching threshold "text_threshold" = 0.25. This indicates the multi-mask segmentation configuration parameters for SAM2, which may include segmentation prompt format, multi-mask output switch, model weights, and other parameter configurations. This describes the rules for calculating the minimum circumscribed circle and area, used for calculating the rim openwork ratio. The function is not a single deep model output, but is composed of open vocabulary localization, cue-based segmentation, and geometric formula calculation, thus possessing replaceable and verifiable engineering features.

[0132] Among these methods, Grounding DINO can be used to guide rim detection using textual prompts. Computer devices can use an open-vocabulary detection model to detect rims on an input image, locating the rim region within the image, outputting multiple candidate bounding boxes, and determining the target bounding box based on these candidate boxes. The Grounding DINO open-vocabulary object detection model is employed, using natural language text as prompts. In rim detection, the computer device can first verify that the input image is in uint8 format and that the pixel range is within [0, 255], then call Grounding DINO to obtain candidate bounding boxes, confidence scores, and text matching scores. Each candidate bounding box can be ranked according to its confidence score. Sort, select , For the first The confidence level of each candidate bounding box. For the target bounding box, To take the maximum value. If the confidence level of the target bounding box... If a low confidence state is output as a prompt, 0.6 is the first confidence threshold (RIM_CONFIDENCE_THRESHOLD) to ensure that subsequent SAM2 segmentation is performed only on reliable rim regions. In some embodiments, the open vocabulary detection model can also use other open vocabulary detection models such as OWL-ViT, CLIP+RPN (Region Proposal Network), or be replaced with a specially trained rim detection model.

[0133] In the wheel rim detection process, the input includes the original image of the wheel rim, which can be in RGB format and of any resolution. The prompts can include the text "wheel rim," "alloy wheel," or "car rim," and the appropriate prompt can be dynamically selected based on the detection scenario. A bounding box confidence threshold of box_threshold=0.35 means only bounding boxes with a confidence score ≥0.35 are retained, and a text matching threshold of text_threshold=0.25 means only bounding boxes with a text matching score ≥0.25 are retained. The output of the wheel rim detection process can include a list of candidate bounding boxes in (center point x, center point y, width, height) format, along with the confidence score of each candidate bounding box. The computer can select the candidate bounding box with the highest confidence score from all candidate bounding boxes as the target bounding box for subsequent segmentation. Wheel rim detection based on an open vocabulary detection model eliminates the need for pre-fixed wheel rim category numbers or fine-tuning with a local data model. It can locate the wheel rim region using text prompts, making it suitable for scenarios with only image input and a lack of dedicated detection datasets.

[0134] In SAM2-based wheel rim image segmentation and multi-candidate mask generation, the computer device can use the obtained target bounding box as input cue to the SAM2 image segmentation model to generate multiple candidate segmentation masks for the wheel rim region. (Target bounding box) It can be converted from (cx, cy, w, h) to Then input the SAM2 predictor and call predict(point_coords=None, point_labels=None, box= The `multimask_output=True` function performs multimask segmentation and generates... ;in, For the target bounding box The minimum x-coordinate value, For the target bounding box The minimum ordinate value, For the target bounding box The maximum x-coordinate value, Target bounding box The maximum ordinate value; point_coords=None and point_labels=None indicate that no point cues are provided to the SAM2 model; multimask_output=True indicates that the multimask output mode is enabled, that is, multiple candidate segmentation masks with different semantic levels are generated under the same target bounding box cues, which are used to handle segmentation ambiguity scenarios; For the first One candidate segmentation mask, For SAM2 targeting the first The confidence level is determined by each candidate segmentation mask. By retaining multiple candidate segmentation masks, semantic ambiguities in the rim, inner ring, spokes, and cutout areas can be further filtered during the geometric evaluation stage. In some embodiments, lightweight variants of models such as SAM (Segment Anything Model), EfficientSAM, MobileSAM, and FastSAM, or specially trained instance segmentation models (such as Mask R-CNN) can also be used.

[0135] In image segmentation model-based segmentation, a cue-driven general segmentation model can be used. Without specifically training a rim segmentation model, bounding box cues can be directly used to generate candidate segmentation masks for the rim. For example, the SAM2 model can be used to encode features of the input image using the `set_image()` interface, extracting multi-scale feature representations without re-encoding; the same image only needs to be encoded once. The SAM2 model can convert the center-format bounding box (cx, cy, w, h) output by Grounding DINO into the corner format required by the SAM2 model. The SAM2 model can call the SAM2 model's `predict()` interface and set `multimask_output=True` (enabling multi-mask output mode) to generate multiple candidate segmentation masks, such as three candidate segmentation masks. Each candidate segmentation mask can be accompanied by a SAM2 confidence score. The computer device can sort the candidate segmentation masks in descending order of confidence score and determine the candidate segmentation mask with the highest confidence score as the target segmentation mask. In some embodiments, if the confidence score of each candidate bounding box is less than a preset first confidence threshold (e.g., 0.6), it is considered that a sufficiently reliable rim region has not been detected in the input image, and a low confidence warning is returned.

[0136] like Figure 5As shown, the computer device can input the image of the wheel rim and the prompt words into an open vocabulary detection model to detect the wheel rim and obtain the target bounding box. The target bounding box is then input into an image segmentation model for segmentation, resulting in multiple candidate segmentation masks, such as candidate segmentation mask 1 and candidate segmentation mask 2. Candidate segmentation mask 1 has a confidence score of 0.92, indicating that the SAM2 model considers the segmentation result highly reliable, accurately distinguishing between the solid areas of the spokes and the hollow areas of the wheel rim. Accurate hollowing rate calculation can be achieved based on candidate segmentation mask 1. Candidate segmentation mask 2 has a confidence score of 0.78, indicating that the SAM2 model has doubts about the reliability of the segmentation result, suggesting that it incorrectly fills the hollow areas between the spokes as solid, thus assuming the wheel rim is entirely filled. Accurate hollowing rate calculation cannot be achieved based on candidate segmentation mask 2.

[0137] In the process of calculating the cutout rate for each candidate segmentation mask, such as Figure 6 As shown, it includes:

[0138] Step S601: Determine candidate segmentation masks. The computer device can iterate through each candidate segmentation mask.

[0139] Step S602, extract the outer contour. The computer device can use findContours (...) The RETR_EXTERNAL function extracts the first... The candidate segmentation mask corresponds to the outer contour with the largest area. .

[0140] Step S603: Determine the minimum circumcircle. The computer device can use minEnclosingCircle( Determine the first The outer contour of a candidate segmentation mask The `minEnclosingCircle()` function calculates the minimum enclosing circle of a given set of points using the Welzl algorithm, with a time complexity of O(n).

[0141] Step S604: Determine the center and radius. The computer device can determine the center of the smallest circumcircle. and radius .

[0142] Step S605: Generate a circular reference mask. The computer device can construct a circular reference mask circle_mask corresponding to the smallest circumcircle, such that the area inside the circle in the reference mask circle_mask is filled with 1s, and the rest are 0s.

[0143] Step S606: Determine the total area of ​​the circumcircle (reference area). The computer device can calculate the total area of ​​the circumcircle based on the geometric parameters of the smallest circumcircle, thus obtaining the reference area. Alternatively, the computer device can count the number of pixels in the reference mask circle_mask to obtain the reference area.

[0144] Step S607: Determine the intersection area. The computer device can determine the solid area (solid) of the intersection between the solid region of the rim and the minimum circumcircle in the candidate segmentation mask.

[0145] Step S608: Calculate the rim openwork ratio. The computer equipment can calculate the rim openwork ratio based on (reference base area - intersection area) / reference base area.

[0146] For each candidate mask Computer equipment can use findContours ( Extract the outer contour using RETR_EXTERNAL and select the outer contour with the largest area. Execute minEnclosingCircle( To determine the minimum circumcircle (i.e., the envelope geometry), the center of the circle is obtained. and radius Computer devices can construct circular reference masks. The reference area formula is as follows The solid area of ​​the wheel rim is written as The actual perforation rate is written as .in, For the first The rim cutout ratio is determined by a candidate segmentation mask.

[0147] In the hollowing ratio calculation process, the computer equipment can perform the `findContours` operation on the binarized candidate segmentation mask (0 / 1 matrix) and use the `RETR_EXTERNAL` mode to extract only the outermost contour, selecting the contour with the largest area as the outer contour of the rim. The computer equipment can then perform the `minEnclosingCircle` algorithm on the outer contour of the rim, outputting the coordinates of the center of the smallest circumcircle. and radius (Unit: pixels). The smallest circumcircle represents the theoretically complete circular area of ​​the rim's front surface, serving as the reference area for calculating the hollowing-out ratio. The formula for calculating the hollowing-out ratio is as follows:

[0148]

[0149] in, =Minimum circumscribed circle area (in pixels)= (Approximately the sum of the number of pixels in the reference mask); =The area of ​​the intersection of the candidate segmentation mask and the reference mask (i.e., the number of pixels in the solid rim portion) = (mask&circle_mask).sum(). For the circular reference mask circle_mask, it can be... Center of the circle The radius is drawn, and the solid area of ​​the rim can be calculated by bitwise AND (&) with the candidate segmentation mask using the reference mask circle_mask.

[0150] For each candidate segmentation mask, we can obtain {center, radius, total_area, solid_area, hollowness_rate, score}, where center is the coordinate of the center of the reference mask circle_mask or the smallest circumcircle; radius is the radius of the reference mask circle_mask or the smallest circumcircle; and total_area is the reference base area. solid_area is the solid area of ​​the wheel rim. Hollowness_rate is the calculated hollowness rate of the rim; score is the confidence level of the candidate segmentation mask. Among them, the minimum circumcircle method uses the outer circle of the rim's outer contour as the reference area. Compared with directly using the area of ​​the rectangular bounding box, it is closer to the geometric features of the rim's front view and facilitates the formation of a unified definition of hollowness rate.

[0151] In some embodiments, the envelope geometry may employ a minimum circumscribed circle or a minimum circumscribed ellipse, which provides better adaptability to elliptical rim shooting angles (non-orthographic views) and is suitable for scenarios where the rim shooting angle is skewed (non-orthographic view, slight perspective distortion). In some embodiments, the envelope geometry may also include a convex hull, thereby enabling its application in calculating the hollowness ratio of non-circular rims.

[0152] For outlier detection and automatic candidate mask backoff mechanisms, such as Figure 7 As shown, it includes:

[0153] Step S701: Obtain multiple candidate segmentation masks.

[0154] Step S702: Determine the rim cutout ratio of the target segmentation mask with the highest confidence level. The computer device can determine the candidate segmentation mask with the highest confidence level as the target segmentation mask and determine the rim cutout ratio of the target segmentation mask.

[0155] Step S703: Determine whether the rim cutout rate of the target segmentation mask is within the cutout rate range. The computer device can determine whether the rim cutout rate of the target segmentation mask is within the cutout rate range. If so, proceed to step S704; if the rim cutout rate of the target segmentation mask is not within the cutout rate range, proceed to step S705.

[0156] Step S704: Determine the cutout rate detection result based on the rim cutout rate of the target segmentation mask. The computer equipment can obtain the cutout rate detection result based on the rim cutout rate of the target segmentation mask. The cutout rate detection result may include the target cutout rate detected for the rim.

[0157] Step S705: Filter intermediate segmentation masks from the remaining segmentation masks. The computer device can filter intermediate segmentation masks from the remaining segmentation masks excluding the target segmentation mask, for example, by filtering based on whether the confidence level exceeds a second confidence threshold or the cutout rate is within the cutout rate range.

[0158] Step S706: Determine whether the intermediate segmentation mask is empty. The computer device can determine whether the filtered intermediate segmentation mask is empty. If it is empty, step S704 can be executed; if the intermediate segmentation mask is not empty, step S707 can be executed.

[0159] Step S707: Determine the cutout rate detection result based on the intermediate segmentation mask. The computer equipment can determine the target cutout rate based on the intermediate segmentation mask, for example, by prioritizing confidence level and then rim cutout rate to determine the target cutout rate.

[0160] In scenarios with multiple candidate segmentation masks, the target segmentation mask with the highest confidence level sometimes fills in the spoke gaps of the wheel rim as foreground, resulting in an abnormally low calculated hollowness ratio. Therefore, an anomaly handling mechanism can be implemented: a minimum hollowness ratio can be pre-configured. Maximum perforation rate Second confidence threshold If the target segmentation mask with the highest confidence level has a rim hollowness rate... satisfy If so, it is directly adopted, that is, based on the rim hollowness ratio of the target segmentation mask. Obtain the cutout rate detection result; otherwise, construct a set of intermediate segmentation masks. and in (middle press) , Select the final mask; where, For the first One candidate segmentation mask, For the first The confidence level of each candidate segmentation mask. For the first The rim cutout ratio is determined by a candidate segmentation mask. If If empty, the rim cutout ratio of the target segmentation mask is retained and an anomaly flag is output. This anomaly handling mechanism can transform a reasonable cutout ratio range into an executable candidate backoff formula.

[0161] In some embodiments, the computer device can perform a reasonableness check on the rim hollowing ratio calculated by the target segmentation mask with the highest confidence level. A reasonable hollowing ratio range is defined as 3% to 47% (based on industry rim design experience, the spoke ratio of solid rims is no less than 3%, and high hollowing ratio rims do not exceed 47%). If 47% ≥ hollowing ratio ≥ 3% (normal situation), the calculation result of the mask is directly adopted; if the hollowing ratio < 3% or 47% < hollowing ratio (abnormal situation), the candidate mask fallback process is triggered. The computer device can filter the set of intermediate segmentation masks whose hollowing ratio falls within the reasonable range [3%, 47%] from all candidate segmentation masks with a confidence level ≥ 0.5.

[0162] In the set of intermediate segmentation masks, the candidate segmentation mask is selected based on its confidence score as the primary sorting key and its hollowness rate as the secondary sorting key, to determine the hollowness rate. Alternatively, selection can be based on a combination of (score, hollowness_rate) priority, meaning selection can be made first by the confidence score and then by the hollowness rate.

[0163] In some embodiments, if none of the candidate segmentation masks meet the reasonable range, the cutout rate detection result is obtained based on the cutout rate of the target segmentation mask, and the detection confidence is marked in the output for user reference.

[0164] The component hollowing rate determination method provided in this application can directly locate the rim area from the rim image and calculate the hollowing rate, avoiding the need for manual drawing of hollowing areas piece by piece on drawings or photographs, thus reducing the workload of manual segmentation. Even in scenarios with only photographs, renderings, or screenshots and no 3D model, a uniformly defined hollowing rate result can still be obtained, suitable for early evaluation and batch analysis, and applicable to image scenarios without CAD (Computer-Aided Design) models. Furthermore, through the combined calculation method of "segmentation mask + envelope circle area," different rim styles can be uniformly mapped to the same type of area ratio index, facilitating subsequent weight prediction and cost analysis. The component hollowing rate determination method provided in this application adds a reasonable interval judgment and backtracking selection mechanism to address semantic ambiguity caused by multiple mask outputs, which helps reduce the misuse of abnormally low hollowing rate results. Component detection, image segmentation, and hollowing rate calculation are all independent of each other, allowing for easy replacement of implementation methods according to different hardware environments or business needs. In addition, the output results can include the candidate segmentation mask and the geometric parameters of the circumcircle, which facilitates manual review of whether the cutout rate results are reasonable and makes it easy to perform visual verification.

[0165] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0166] Based on the same inventive concept, this application also provides a component hollowing rate determination device for implementing the component hollowing rate determination method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more component hollowing rate determination device embodiments provided below can be found in the limitations of the component hollowing rate determination method above, and will not be repeated here.

[0167] In one exemplary embodiment, such as Figure 8As shown, a component hollowness rate determination device 800 is provided, including: a component detection module 801, a segmentation mask generation module 802, a hollowness rate determination module 803, and a detection result acquisition module 804, wherein:

[0168] The component detection module 801 is used to input image data and preset prompts into a pre-trained component recognition model to obtain the target bounding box output by the component recognition model; the target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data;

[0169] The segmentation mask generation module 802 is used to segment the image of the component in the image data according to the target bounding box, and obtain multiple candidate segmentation masks of the component and the confidence level of each candidate segmentation mask;

[0170] The cutout rate determination module 803 is used to determine the cutout rate of the component corresponding to each candidate segmentation mask, and obtain a candidate cutout rate set;

[0171] The detection result acquisition module 804 is used to determine the target cutout rate from the candidate cutout rate set based on the confidence level.

[0172] In some embodiments, the open vocabulary detection model includes an open vocabulary detection model and a component detection module 801, which is further configured to input image data and preset prompt words into the pre-trained open vocabulary detection model to obtain multiple candidate bounding boxes output by the open vocabulary detection model and the confidence level of each candidate bounding box; and to determine the target bounding box from each candidate bounding box based on the confidence level of each candidate bounding box.

[0173] In some embodiments, the constraints on the open vocabulary detection model include at least one of the following: a bounding box confidence threshold, used to constrain the range of confidence values ​​of the candidate bounding boxes output by the open vocabulary detection model; and a text matching threshold, used to constrain the range of text matching values ​​of the candidate bounding boxes output by the open vocabulary detection model, wherein the text matching degree is used to quantify the degree of association between the image region marked by the candidate bounding box and the prompt word.

[0174] In some embodiments, the component hollowing rate determination device 800 further includes a prompting module, which generates a prompting message when the confidence scores of each candidate bounding box are all less than a preset first confidence threshold, to indicate that the image data does not support hollowing rate determination.

[0175] In some embodiments, the segmentation mask generation module 802 is further configured to extract multi-scale features of components from image data; and to segment the image of components in the image data based on multi-scale features and target bounding boxes using a pre-trained image segmentation model to obtain multiple candidate segmentation masks and the confidence level of each candidate segmentation mask.

[0176] In some embodiments, the cutout rate determination module 803 is further configured to: determine the envelope geometry based on the outer contour of the candidate segmentation mask for each candidate segmentation mask; determine the intersection area between the solid region of the component in the candidate segmentation mask and the envelope geometry according to the candidate segmentation mask and the envelope geometry; determine the reference area of ​​the envelope geometry; and obtain the cutout rate of the component corresponding to the candidate segmentation mask according to the intersection area and the reference area; and obtain a set of candidate cutout rates according to the cutout rates corresponding to each candidate segmentation mask.

[0177] In some embodiments, the detection result acquisition module 804 is further configured to determine the target segmentation mask from each candidate segmentation mask based on the confidence level; and if the cutout rate corresponding to the target segmentation mask is within a preset cutout rate range, obtain the target cutout rate of the component based on the cutout rate of the target segmentation mask.

[0178] In some embodiments, the detection result acquisition module 804 is further configured to determine each remaining segmentation mask when the cutout rate of the target segmentation mask is not within the cutout rate range; the remaining segmentation masks include candidate segmentation masks other than the target segmentation mask among each candidate segmentation mask; determine intermediate segmentation masks from each remaining segmentation mask according to their respective confidence level and cutout rate; the confidence level of the intermediate segmentation mask exceeds a preset second confidence level threshold and the cutout rate of the intermediate segmentation mask is within the cutout rate range; when the intermediate segmentation mask is not empty, obtain the target cutout rate of the component based on the respective confidence level and cutout rate of each intermediate segmentation mask; when the intermediate segmentation mask is empty, obtain the target cutout rate of the component according to the cutout rate of the target segmentation mask.

[0179] Each module in the aforementioned component hollowness determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0180] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores various data involved in the method for determining the hollowness ratio of components. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for determining the hollowness ratio of components.

[0181] Those skilled in the art will understand that Figure 9 The structure shown is a block diagram of a partial structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0182] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0183] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0184] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0185] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0186] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program mentioned can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0187] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0188] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the hollowness ratio of a component, characterized in that, The method includes: Image data and preset prompts are input into a pre-trained component recognition model to obtain the target bounding box output by the component recognition model; the target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data; Based on the target bounding box, the image of the component in the image data is segmented to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask; The cutout ratio of each component corresponding to each candidate segmentation mask is determined to obtain a candidate cutout ratio set; Based on the confidence level, the target cutout rate is determined from the set of candidate cutout rates.

2. The method according to claim 1, characterized in that, The component recognition model includes an open vocabulary detection model. Image data and preset prompts are input into the pre-trained component recognition model to obtain the target bounding box output by the model, including: Image data and preset prompts are input into the pre-trained open vocabulary detection model to obtain multiple candidate bounding boxes and the confidence score of each candidate bounding box output by the open vocabulary detection model. The target bounding box is determined from each of the candidate bounding boxes based on the confidence level of each candidate bounding box.

3. The method according to claim 2, characterized in that, The constraints on the open vocabulary detection model include at least one of the following: The bounding box confidence threshold is used to constrain the range of confidence values ​​for the candidate bounding boxes output by the open vocabulary detection model. The text matching degree threshold is used to constrain the range of text matching degree values ​​of the candidate bounding boxes output by the open vocabulary detection model. The text matching degree is used to quantify the degree of association between the image region marked by the candidate bounding box and the prompt word.

4. The method according to claim 2, characterized in that, The method further includes: If the confidence level of each candidate bounding box is less than a preset first confidence threshold, a prompt message is generated to indicate that the image data does not support the determination of the cutout rate.

5. The method according to claim 1, characterized in that, Based on the target bounding box, the image of the component in the image data is segmented to obtain multiple candidate segmentation masks for the component and the confidence level of each candidate segmentation mask, including: Extract multi-scale features of the components from the image data; A pre-trained image segmentation model is used to segment the images of the components in the image data based on the multi-scale features and the target bounding box, thereby obtaining multiple candidate segmentation masks and the confidence level of each candidate segmentation mask.

6. The method according to claim 1, characterized in that, The cutout ratio of each component corresponding to each candidate segmentation mask is determined to obtain a candidate cutout ratio set, including: For each candidate segmentation mask, the envelope geometry is determined based on the outer contour of the candidate segmentation mask. Based on the target candidate segmentation mask and the envelope geometry, determine the intersection area between the solid region of the component in the target candidate segmentation mask and the envelope geometry; Determine the reference area of ​​the envelope geometry, and obtain the cutout ratio of the component corresponding to the candidate segmentation mask based on the intersection area and the reference area; Based on the cutout ratio corresponding to each candidate segmentation mask, a candidate cutout ratio set is obtained.

7. The method according to any one of claims 1 to 6, characterized in that, Based on the confidence level, the target cutout rate is determined from the candidate cutout rate set, including: Based on the confidence level, the target segmentation mask is determined from each of the candidate segmentation masks; If the cutout rate corresponding to the target segmentation mask is within a preset cutout rate range, the target cutout rate of the component is obtained based on the cutout rate of the target segmentation mask.

8. The method according to claim 7, characterized in that, The method further includes: If the cutout rate of the target segmentation mask is not within the cutout rate range, each remaining segmentation mask is determined; the remaining segmentation masks include the candidate segmentation masks other than the target segmentation mask among each candidate segmentation mask; Based on the confidence level and cutout rate of each of the remaining segmentation masks, an intermediate segmentation mask is determined from each of the remaining segmentation masks; the confidence level of the intermediate segmentation mask exceeds a preset second confidence level threshold, and the cutout rate of the intermediate segmentation mask is within the cutout rate range; If the intermediate segmentation mask is not empty, the target cutout rate of the component is obtained based on the confidence level and cutout rate of each intermediate segmentation mask. When the intermediate segmentation mask is empty, the target cutout ratio of the component is obtained based on the cutout ratio of the target segmentation mask.

9. A device for determining the hollowness rate of components, characterized in that, The device includes: The component detection module is used to input image data and preset prompts into a pre-trained component recognition model to obtain the target bounding box output by the component recognition model; the target bounding box marks the area occupied by the component in the image data, and the prompts are used to guide the component recognition model to analyze and infer the area occupied by the component in the image data; The segmentation mask generation module is used to segment the image of the component in the image data according to the target bounding box, and obtain multiple candidate segmentation masks of the component and the confidence level of each candidate segmentation mask; The cutout rate determination module is used to determine the cutout rate of the component corresponding to each candidate segmentation mask, and obtain a candidate cutout rate set; The detection result acquisition module is used to determine the target cutout rate from the candidate cutout rate set based on the confidence level.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images