A cocoon sorting decision model training method based on a multi-modal large model

Abstract

The application discloses a cocoon sorting decision model training method based on a multi-modal large model, collects cocoon visible light images, near-infrared spectra and 3D point clouds and constructs a cocoon topology training atlas, selects defect-free self-checking nodes corresponding to defect nodes in the axial direction, opposite in the circumferential direction and matching in the curvature in the same cocoon, generates counterfactual evidence migration sample groups through modal data exchange, carries out first stage training on the multi-modal large model, forms defect modal migration prototypes according to defect node predicted drop, self-checking node predicted rise and defect position migration amount, and carries out second stage training accordingly, further uses predicted differences before and after sample resetting to form resetting residuals, and carries out weighted iterative training on high residual samples until convergence to obtain the cocoon sorting decision model. The application can enhance the learning ability of the model to real defect evidence and spatial correlation, and improve the accuracy and stability of the sorting decision under different cocoon conditions.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training method for a silkworm cocoon sorting decision model based on a multimodal large model. Background Technology

[0002] Currently, silkworm cocoon sorting typically employs manual inspection, machine vision recognition, and automated classification based on deep learning models. With the development of sensor and artificial intelligence technologies, existing techniques are beginning to combine different types of data, such as visible light images, near-infrared spectroscopy, and 3D point clouds, to characterize the appearance, color, surface texture, morphological structure, and internal or near-surface features of silkworm cocoons. Feature extraction, feature fusion, and quality classification are then performed using neural networks, deep learning models, or multimodal models. Some methods further enhance the model's ability to identify cocoons of different quality grades and defect types by labeling training samples, applying transfer learning, or fine-tuning the model, and providing decision-making support for subsequent automated sorting based on the model's output.

[0003] However, existing model training methods typically rely on statistical differences between different silkworm cocoon samples as the primary learning basis. Different silkworm cocoons exhibit natural individual differences in variety, size, color, silk layer texture, moisture content, and surface curvature, making it easy for the model to learn individual features unrelated to defects as classification criteria. Furthermore, the response mechanisms and spatial distributions of visible light images, near-infrared spectra, and 3D point clouds differ for different defects. Conventional feature fusion training struggles to determine which modality's locally valid evidence drives the model's defect prediction, and it's difficult to verify whether the model's prediction changes accordingly after spatial variations in defect evidence. In addition, existing training methods often lack constraints on the model's predictive consistency during evidence perturbation and evidence recovery processes. This can lead to the model relying on accidental related features or anomalous responses from a single modality. When silkworm cocoon variety, origin, morphology, and collection conditions change, problems such as defect localization shifts, misclassification, and decreased stability in sorting decisions can easily occur.

[0004] Therefore, it is necessary to propose a model training method that can improve the effectiveness, accuracy, and generalization ability of the silkworm cocoon sorting decision model. Summary of the Invention

[0005] This application provides a training method for a silkworm cocoon sorting decision model based on a multimodal large model, in order to improve the accuracy and stability of sorting decisions under different silkworm cocoon conditions.

[0006] This application provides a training method for a silkworm cocoon sorting decision model based on a multimodal large model, including: Collect visible light images, near-infrared spectra, and 3D point clouds of silkworm cocoons, label quality grades, defect types, and defect areas, determine the main axis and waist section of the cocoon from the 3D point cloud, establish axial and circumferential surface nodes and map the three types of data, and generate a silkworm cocoon topology training map. For defective nodes, select non-defective self-control nodes that are axially corresponding, circumferentially opposite, and have a curvature difference below the threshold in the same cocoon. Exchange the data of defective nodes and self-control nodes modally to generate a counterfactual evidence transfer sample group. The first stage of training of the multimodal large model is carried out using the silkworm cocoon topology training graph. The model parameters are updated while keeping the model structure unchanged to obtain the first stage model. The counterfactual evidence transfer sample group is input into the first stage model. The defect mode transfer vector is generated based on the predicted descent of defect nodes, the predicted rise of self-control nodes, and the defect location transfer. Defect mode transfer prototypes are formed according to defect type. The first-stage model was trained using a silkworm cocoon topology training graph, a counterfactual evidence transfer sample set, and a defective mode transfer prototype. The model parameters were updated while keeping the model structure unchanged, so that the defective mode transfer vector approximates the defective mode transfer prototype, thus obtaining the second-stage model. The counterfactual evidence transfer sample group is reset, and the prediction difference of the second-stage model before and after reset is calculated as the reset residual. The counterfactual evidence transfer sample group with reset residual exceeding the threshold is weighted to train the second-stage model. The model parameters are updated until the classification error and reset residual converge, and the cocoon sorting decision model is obtained.

[0007] The beneficial effects of the technical solution provided in this application include: (1) By constructing a silkworm cocoon topology training atlas, the spatial correspondence of visible light images, near-infrared spectra, and 3D point clouds on a unified surface node is realized, thereby improving the consistency of multimodal feature expression. (2) By modally swapping defect nodes and self-control nodes within the same silkworm cocoon, the interference of individual differences in silkworm cocoons on model training is reduced. (3) By using defect modal transfer vectors and defect modal transfer prototypes, the model learns the effective modal evidence corresponding to different defects and their spatial transfer relationships, thereby improving the accuracy of defect identification and localization. (4) By performing weighted iterative training on high-biased samples using reset residuals, the stability of the model to evidence perturbation and recovery is enhanced, thereby improving the generalization ability and reliability of silkworm cocoon sorting decisions under different varieties, morphologies, and collection conditions. Attached Figure Description

[0008] Figure 1 This is a flowchart of a method for training a silkworm cocoon sorting decision model based on a multimodal large model, provided in the first embodiment of this application.

[0009] Figure 2 This is a schematic diagram illustrating the training convergence and reset stability effect of the first embodiment of this application.

[0010] Figure 3 This is a schematic diagram of the electronic device involved in the second embodiment of this application. Detailed Implementation

[0011] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0012] The first embodiment of this application provides a method for training a silkworm cocoon sorting decision model based on a multimodal large model. Please refer to... Figure 1 This figure is a schematic diagram of the first embodiment of this application. The following is in conjunction with... Figure 1 The first embodiment of this application provides a detailed description of a method for training a silkworm cocoon sorting decision model based on a multimodal large model.

[0013] Step S101: Collect visible light images, near-infrared spectra, and 3D point clouds of silkworm cocoons, label the quality grade, defect type, and defect area, determine the main axis and waist section of the cocoon from the 3D point cloud, establish axial and circumferential surface nodes and map the three types of data, and generate a silkworm cocoon topology training map.

[0014] Step S101 is used to convert the visible light information, near-infrared spectral information, and three-dimensional morphological information of the same silkworm cocoon into training data with a unified spatial positional relationship. In practice, the silkworm cocoon to be collected can be placed on a rotatable support mechanism or a continuous conveying mechanism. Visible light images from multiple perspectives are acquired using an industrial camera, near-infrared spectra associated with the cocoon's surface position are acquired using a near-infrared acquisition device, and 3D point clouds of the cocoon's surface are obtained using a 3D camera, structured light equipment, or laser profilometry. The three types of data are collected from the same silkworm cocoon, and the acquisition perspective, scanning position, and equipment calibration parameters are saved, enabling data from different modalities to be reconstructed to the same cocoon surface position.

[0015] The collected silkworm cocoons are labeled with their quality grade, defect type, and defect area. The quality grade is a sorting category for whole cocoons; the defect type classifies abnormalities such as yellow spots, thin skin, maggot holes, bark marks, and double cocoons; the defect area is the actual spatial extent of the defect on the cocoon surface. The defect area can first be manually drawn in a visible light image and then projected onto a 3D point cloud using camera calibration. For defects primarily identified based on near-infrared spectroscopy, the corresponding 3D point cloud area can be determined based on the near-infrared sampling location.

[0016] After obtaining the 3D point cloud, a principal direction analysis is performed on the 3D point cloud after removing outliers. The direction with the greatest spatial dispersion is determined as the cocoon principal axis direction, and the position of the cocoon principal axis is determined by the geometric center of the point cloud. The cocoon principal axis is used to characterize the length direction of the silkworm cocoon. An axial position reference is established by taking one end of the cocoon principal axis as the axial starting point and the other end as the axial ending point.

[0017] The cocoon waist section is a reference section perpendicular to the main axis of the cocoon, located in the middle of the cocoon, and used to establish a circumferential positional reference. Multiple perpendicular sections can be selected along the main axis of the cocoon in the middle of the cocoon; the section with a continuous profile, complete point cloud, and the largest lateral dimension is chosen as the cocoon waist section. For approximately spindle-shaped cocoons, the section with the largest lateral dimension near the midpoint of the main axis can also be directly selected.

[0018] After determining the cocoon's main axis and the cocoon waist section, axial and circumferential surface nodes are established. These axial and circumferential surface nodes are local surface regions divided according to the cocoon's axial position and its circumferential position around the main axis. The axial position can be normalized from the axial start point to the axial end point and divided into multiple axial intervals, for example, twenty axial intervals. The circumferential position uses a fixed reference direction within the cocoon waist section as the zero-angle direction, dividing the area around the main axis of the cocoon into multiple circumferential intervals, for example, twelve circumferential intervals. The fixed reference direction can be determined by projecting the X-axis of the 3D point cloud acquisition coordinate system onto the cocoon waist section; when the X-axis is approximately parallel to the cocoon's main axis and cannot form a stable projection, the projection direction of the Y-axis of the acquisition coordinate system onto the cocoon waist section is used, and the direction of increasing the circumferential angle is determined according to a predetermined right-hand direction, thereby ensuring a consistent circumferential position reference in the same or repeated processing.

[0019] A surface node is defined by an axial interval and a circumferential interval, with each surface node uniquely identified by its axial and circumferential interval numbers. The axial positional relationship between any two surface nodes is determined by the difference in their axial interval numbers, and the circumferential positional relationship is determined by the shortest interval between the two circumferential interval numbers along the circumferential direction. This establishes a measurable axial and circumferential positional relationship between surface nodes, providing a spatial coordinate basis for subsequently determining the migration amount of defect locations.

[0020] When mapping the three types of data to axial and circumferential surface nodes, 3D point clouds are used as spatial references. For visible light images, 3D surface points are projected onto the corresponding images based on camera intrinsic and extrinsic parameters, and color, brightness, and texture features are extracted. For near-infrared spectra, near-infrared spectra are assigned to corresponding surface nodes based on scanning trajectories, spot positions, or pre-defined spatial correspondences. The coordinates, normal vectors, local curvature, and surface undulation features of the 3D point cloud are directly assigned to the corresponding surface nodes.

[0021] For example, dividing a silkworm cocoon into twenty axial intervals and twelve circumferential intervals results in 240 surface nodes. If a certain macula region is located in the ninth axial interval and the third circumferential interval, then the visible light characteristics, near-infrared spectrum, and 3D morphological characteristics corresponding to this region are all stored in that surface node. Surface nodes covered by defective regions are marked as defective nodes, and surface nodes not covered by defective regions are marked as defect-free nodes. Adjacency relationships are established between surface nodes in adjacent axial intervals or adjacent circumferential intervals.

[0022] A cocoon topology training graph is formed by all surface nodes of a single cocoon, the adjacency relationships between surface nodes, the three types of modal data corresponding to each surface node, as well as the quality grade, defect type, and defect region label. After processing multiple labeled cocoons in the same way, a cocoon topology training graph set is formed, which provides a unified data foundation for subsequent self-control node selection, counterfactual evidence transfer sample generation, and multimodal large model training.

[0023] Furthermore, the acquisition of visible light images, near-infrared spectra, and 3D point clouds of silkworm cocoons, labeling quality grades, defect types, and defect areas, determining the cocoon's main axis and waist section from the 3D point cloud, establishing axial and circumferential surface nodes and mapping the three types of data, and generating a silkworm cocoon topology training atlas, including: The main axis of the cocoon, the cross section of the cocoon waist, the two extreme points of the cocoon end, and the main direction of the cocoon waist contour are determined by the 3D point cloud, generating a cocoon positioning reference with a unique axial direction and circumferential zero position; Based on the cocoon positioning benchmark, the axial region and the circumferential region are equally divided according to the cumulative surface area, and the boundary line is adjusted with the minimum geodesic distance deviation between adjacent nodes as a constraint to generate a set of topological nodes with equal surface area. Visible light images and near-infrared spectra are mapped to a set of topological nodes with equal surface area. The effective coverage ratio of each node for the three modes and the intersection coverage ratio are calculated. Sampling positions with an intersection coverage ratio lower than the threshold are removed to generate a cross-modal co-certification node set. The defect region is mapped to the cross-modal co-certification node set. The nodes that contain both defect and non-defect regions are subdivided along the defect boundary, and the adjacency edges are reconstructed according to the geodesic adjacency relationship of the nodes before and after subdivision, generating a defect boundary-preserving node set. By maintaining the three modalities of the node set at the defect boundary, node location, effective coverage information, quality level, defect type, defect region label, and adjacency relationship association, a cocoon topology training graph is generated.

[0024] This implementation method is used to establish a stable spatial positioning benchmark based on the three-dimensional morphology of a single silkworm cocoon, and to make visible light images, near-infrared spectra and 3D point clouds form an effective correspondence on a unified topological position on the surface of the silkworm cocoon, thereby obtaining a silkworm cocoon topology training map that can maintain defect boundaries and cross-modal spatial consistency.

[0025] After obtaining the 3D point cloud of the silkworm cocoon, the main axis and waist section of the cocoon are determined as described above, and two end poles are further determined. The end poles are the surface points with the largest projected distances along the main axis of the cocoon in both the positive and negative directions, or the end center points obtained by fitting adjacent end points, located at opposite ends of the cocoon's length. To ensure that the same cocoon obtains a unique axial direction in different processing stages, the axial start and end points can be determined based on the local geometric features near the two cocoon ends. For example, the average cross-sectional area within a preset length range of the two cocoon ends is calculated, and the end with the smaller average cross-sectional area is determined as the axial start point, and the other end as the axial end point. When the difference in the average cross-sectional area between the two ends is less than a preset threshold, the axial start point can be determined according to the coordinate order of the two end poles in the 3D point cloud acquisition coordinate system, thereby avoiding random flipping of the axial numbering.

[0026] The principal direction of the cocoon waist contour is used to determine the circumferential zero position. A closed-point cloud contour is extracted from the cocoon waist cross-section. The principal direction of the contour points within the cross-section is analyzed, and the direction with the greatest contour dispersion is taken as the principal direction of the cocoon waist contour. Its positive direction is determined according to the angle between this direction and the preset positive direction of the acquisition coordinate system. The position on the positive side of the intersection of the principal direction of the cocoon waist contour and the cocoon waist contour is determined as the circumferential zero position. The circumferential position is incremented around the main axis of the cocoon body according to a preset rotation direction. The main axis of the cocoon body, the axial starting point, the axial ending point, the cocoon waist cross-section, and the circumferential zero position together constitute the cocoon body positioning reference, ensuring that each cocoon has a uniquely determined axial direction and circumferential position reference.

[0027] Based on the cocoon positioning datum, the cocoon surface is topologically divided using equal surface area. First, the reconstructed cocoon surface is triangulated, and the area of ​​each triangular facet is accumulated from the axial starting point to the axial ending point. When the accumulated area reaches a preset equal division ratio of the entire cocoon surface area, the axial boundary position is determined, thus forming multiple axial regions with approximately equal surface areas. Then, within each axial region, starting from the circumferential zero point, the surface area is accumulated along a preset circumferential direction, and multiple circumferential regions are divided in the same way. The intersection of the axial and circumferential regions constitutes the initial topological node. Using accumulated surface area for division reduces the impact of differences in length, diameter, and local swelling degree among different cocoons on the actual coverage area of ​​the nodes.

[0028] After the initial partitioning, the boundary line is further adjusted based on the geodesic distances of adjacent nodes. The geodesic distance between adjacent nodes is the shortest path length along the reconstructed surface of the cocoon between the geometric centers of two adjacent nodes. For nodes in the same axial layer or adjacent axial layers, the difference between the geodesic distance of each adjacent node and the average geodesic distance of the region is calculated, and the candidate boundary line is moved to reduce the overall value of the difference in geodesic distances between all adjacent nodes. In actual implementation, the boundary line can be moved gradually within a preset range on both sides of the initial boundary line, and the geodesic distances of adjacent nodes are recalculated each time. When further movement cannot further reduce the overall deviation, the final boundary position is determined. For example, if the geodesic distances of adjacent nodes after the initial partitioning are 3.2 mm, 5.1 mm, 3.4 mm, and 5.0 mm, after boundary adjustment they become 4.0 mm, 4.2 mm, 3.9 mm, and 4.1 mm, then the adjusted topological nodes have a more uniform spatial scale on the real cocoon surface. This forms a set of topological nodes with equal surface area.

[0029] Subsequently, using 3D point clouds as a spatial reference, visible light images and near-infrared spectra are mapped to a set of topological nodes with equal surface areas. The 3D surface position corresponding to each pixel in the visible light image is determined based on camera calibration parameters, while the corresponding 3D surface position in the near-infrared spectrum is determined based on the scanning trajectory, sampling spot position, and device calibration relationship. For each topological node, the area covered by effective visible light data, effective near-infrared spectral data, and effective 3D point cloud data within the node's actual surface area is calculated and divided by the node's total surface area to obtain the effective coverage ratio for the corresponding mode. Invalid data includes data that exceeds the sensor's effective field of view, is occluded, has signal saturation, has a signal-to-noise ratio lower than the preset requirement, or cannot be spatially mapped.

[0030] The intersection coverage ratio represents the proportion of the surface area within the same topological node that simultaneously possesses valid data from visible light images, near-infrared spectra, and 3D point clouds, relative to the node's total surface area. Specifically, the node surface can be divided into uniform microgrids. Only when a grid location simultaneously possesses all three types of valid data is it marked as an intersection valid location, and then the coverage area of ​​the intersection valid location is calculated. For example, if a node has a total surface area of ​​20 square millimeters, with visible light, near-infrared spectra, and 3D point cloud valid coverage areas of 19 square millimeters, 16 square millimeters, and 20 square millimeters respectively, and the shared valid coverage area of ​​the three types of data is 15 square millimeters, then the intersection coverage ratio of this node is 0.75. Sampling locations with intersection coverage ratios below a preset threshold are discarded, retaining only sampling locations with stable three-modal shared coverage. The retained nodes constitute a cross-modal co-evidence node set. Cross-modal co-evidence refers to the ability to simultaneously obtain three types of valid observation data at the same surface location; it does not imply that the three types of data must produce the same defect judgment.

[0031] Pre-labeled defect regions are mapped to a cross-modal co-identification node set according to spatial calibration relationships. When a node is entirely within a defect region or entirely within a defect-free region, its node structure remains unchanged; when a node contains both defect and defect-free regions, it is identified as a cross-boundary node. For cross-boundary nodes, the node is locally subdivided based on the intersection of the defect region boundary and the node surface, ensuring that each subdivided node contains only defect or defect-free regions as much as possible. Subdivision only occurs at cross-boundary nodes and does not change the original axial and circumferential topological positions of other nodes.

[0032] After node subdivision, the adjacency relationships between nodes are redefined based on the reconstructed cocoon surface. When two subdivided nodes have a continuous common boundary on the cocoon surface, an adjacency edge is established, and the geodesic distance between the geometric centers of the two nodes is recorded. The adjacency edges between the original cross-boundary node and surrounding nodes are redistributed according to the actual common boundary after subdivision. This preserves both the defect region boundary and the topological continuity of the cocoon surface, forming a defect boundary-preserving node set.

[0033] Finally, the visible light data, near-infrared spectral data, 3D point cloud data, axial and circumferential positions, effective coverage ratios of the three modalities, intersection coverage ratios, quality grades, defect types, defect region labels, and node adjacency relationships corresponding to each node in the defect boundary preservation node set are stored as a single cocoon identifier, forming a topology training graph for each cocoon. Multiple cocoons in the training samples are processed in the same way, forming a cocoon topology training graph set. This provides training data with a unified spatial scale and effective cross-modal coverage for subsequent selection of defect nodes and self-control nodes within the same cocoon, counterfactual evidence transfer, and model training.

[0034] Step S102: For defective nodes, select non-defective self-control nodes that are axially corresponding, circumferentially opposite, and have a curvature difference below the threshold in the same cocoon. Exchange the data of defective nodes and self-control nodes modally to generate a counterfactual evidence transfer sample group.

[0035] Step S102 is used to construct counterfactual training data with clear evidence transfer relationships within the same silkworm cocoon. After forming the silkworm cocoon topology training atlas in step S101, the surface nodes marked as defective nodes are processed one by one, and self-control nodes are determined among the defect-free nodes in the same silkworm cocoon. The self-control nodes are surface nodes that are comparable to the defective nodes in terms of spatial location and local three-dimensional morphology within the cocoon and are not covered by the defective area, and are used to provide normal state data corresponding to the defective nodes. Using nodes within the same silkworm cocoon as self-control nodes can reduce the impact of individual differences in variety, color, size, silk texture, and moisture content of different silkworm cocoons on subsequent evidence transfer analysis.

[0036] When selecting a self-control node, it is preferable that the self-control node simultaneously meets three conditions: being in the same axial interval as the defective node, being in the opposite circumferential interval, and having a curvature difference below a threshold. This ensures the axial correspondence, circumferential opposition, and curvature matching relationships. Axial correspondence means that the self-control node and the defective node are located in the same axial interval. Circumferential opposition means that the self-control node and the defective node are half a circumference apart around the cocoon's main axis. For the case where the circumference is divided into twelve circumferential intervals in step S101, for example, if the defective node is located in the third circumferential interval, then its circumferential opposition interval is the ninth circumferential interval. As an optional implementation, when there is no defect-free node that simultaneously meets the above strict conditions, the axial range can be widened to a preset adjacent axial interval, or the circumferential range can be widened to an adjacent circumferential interval of the opposition interval during the model training data expansion process. This widening method is used to supplement training samples and does not change the implementation method of preferentially using strict correspondence and strict opposition relationships to determine the self-control node.

[0037] Curvature is used to characterize the degree of curvature of the local cocoon surface where a surface node is located. The node curvature can be obtained from the normal variation of the surface node and its adjacent 3D points or from the local surface fitting results, and the absolute difference between the curvature of a defective node and the curvature of a candidate defect-free node is used as the curvature difference. The curvature difference threshold can be predetermined based on the curvature fluctuation range of normal cocoons at the same axial position in the training set. To reduce the curvature scale differences caused by different cocoon sizes, normalized curvature can also be used. Specifically, the curvature of each node can be linearly mapped to the range of 0 to 1 according to the minimum and maximum curvature values ​​in the same axial interval of the training set, ensuring that the defective nodes and candidate defect-free nodes to be compared use the same mapping benchmark. For example, if the normalized curvature of a defective node is 0.36, the normalized curvature of a candidate defect-free node is 0.39, and the curvature difference threshold is 0.05, then the curvature difference is 0.03, which is less than the threshold, and the candidate defect-free node satisfies the curvature condition. If another candidate defect-free node has a normalized curvature of 0.45, then the curvature difference is 0.09, which does not satisfy the curvature condition. When multiple candidate defect-free nodes simultaneously satisfy the condition, the candidate defect-free node with the smallest curvature difference is selected as the self-control node.

[0038] After identifying defective nodes and self-control nodes, the data of defective nodes and self-control nodes are swapped modally. The modal swapping refers to exchanging only one type of data from three modalities—visible light image, near-infrared spectrum, and 3D point cloud—at each exchange. The data from the other two modalities that are not swapped retain their original node affiliations, thereby changing the local defect evidence of one modality individually.

[0039] For example, a defective node contains visible light data Vd, near-infrared spectral data Nd, and 3D point cloud data Pd, while a self-control node contains visible light data Vn, near-infrared spectral data Nn, and 3D point cloud data Pn. When performing visible light mode swapping, Vd and Vn are swapped, while Nd, Nn, Pd, and Pn remain unchanged. When performing near-infrared spectral mode swapping, only Nd and Nn are swapped. When performing 3D point cloud mode swapping, only Pd and Pn are swapped. The swapping does not change the axial or circumferential positions of the defective node and the self-control node, but only changes the data attribution of the specified mode. In the same mode swapping, the defective node loses the original defect evidence for that mode, while the self-control node gains the original defect evidence for that mode. Therefore, it is possible to subsequently observe whether the predicted value of the defective node decreases and whether the predicted value of the self-control node increases, providing corresponding data for step S103 to determine the decrease in the predicted value of the defective node and the increase in the predicted value of the self-control node.

[0040] For 3D point cloud modalities, since the number of points on different surface nodes may vary, the 3D point cloud of each surface node can be encoded into fixed-dimensional 3D morphological features for interchangeability. Alternatively, the number of samples in a local point cloud can be standardized and the local coordinates normalized before interchangeability. Visible light data can use fixed-size image patches of the corresponding surface nodes or encoded visible light features. Near-infrared spectral data can use spectral sequences sampled in a unified band or encoded spectral features to ensure that data of the same modality can be directly interchanged.

[0041] For a defective node and its self-control node, visible light mode swapping, near-infrared spectral mode swapping, and 3D point cloud mode swapping are performed respectively to obtain counterfactual evidence transfer samples corresponding to the three modes. All counterfactual evidence transfer samples generated from the same defective node constitute a counterfactual evidence transfer sample group, and the defective node identifier, self-control node identifier, swapped mode type, and node positional relationship before and after the swap are preserved. The node positional relationship is represented by the axial and circumferential measurable positional relationship established in step S101, that is, the axial and circumferential positional relationships between the defective node and the self-control node are determined by axial interval numbers and circumferential interval numbers, thus providing a unified spatial measurement basis for determining the defect position migration amount in step S103.

[0042] Furthermore, for the defective nodes, in the same silkworm cocoon, non-defective self-control nodes that are axially corresponding, circumferentially opposite, and have a curvature difference below a threshold are selected. The data of the defective nodes and the self-control nodes are then interchanged modally to generate a counterfactual evidence transfer sample group, including: Based on the axial and circumferential interval numbers of the defective nodes, the same axial interval and the opposite circumferential interval are determined. From the intersection area, defect-free nodes with curvature differences below the threshold and where neither the node itself nor its first-level adjacent nodes are covered by the defect area are selected to generate a self-comparison candidate node set. Calculate the difference in normal dip angle, local surface area, and geodesic distance distribution between each defect-free node and defective node in the self-control candidate node set. Normalize each difference and sum them. Select the defect-free node with the smallest sum as the self-control node and generate node pairing relationships. Based on the node pairing relationship, a unified local coordinate system is established on the local tangent plane between the defect node and the self-control node to determine the one-to-one correspondence between the corresponding sampling positions and generate a bidirectional equipotential mapping table. Based on the bidirectional equipotential mapping table, visible light images, near-infrared spectra, and 3D point clouds are equipotentially registered. Each time, only one type of modal data is exchanged while keeping the other two types of modal data, node identifiers, and node topology unchanged, generating three types of counterfactual evidence transfer samples and a reverse reset mapping table. The three types of counterfactual evidence migration samples are associated with node pairing relationships, bidirectional equivalence mapping tables, and reverse reset mapping tables to generate counterfactual evidence migration sample groups.

[0043] In this embodiment, in order to ensure a stable and repeatable spatial comparability between defective nodes and self-control nodes, and to reduce the interference of local morphological differences on subsequent counterfactual evidence migration, a counterfactual evidence migration sample group can be further generated according to this implementation method.

[0044] For each defective node, the axial and circumferential interval numbers of the defective node are determined based on the axial and circumferential surface nodes established in step S101. Surface nodes with the same axial interval number as the defective node are selected, and a corresponding circumferential interval differing from the defective node by half a circumference is determined. For example, if the circumference is divided into twelve intervals, and the defective node is located in the third circumferential interval, the ninth circumferential interval is the corresponding circumferential interval. The area jointly defined by the same axial interval and the corresponding circumferential interval is used as the candidate search range. Surface nodes within the candidate search range must satisfy the condition that the curvature difference with the defective node is less than a preset curvature threshold, and that neither the surface node itself nor its first-level adjacent nodes are covered by the defective region. First-level adjacent nodes are surface nodes that are directly adjacent to the current surface node axially or circumferentially and connected by an edge in the cocoon topology training graph. Defect-free nodes that meet the above conditions form a self-comparison candidate node set.

[0045] Self-reference nodes are further determined based on local 3D morphology within the self-reference candidate node set. The normal tilt angle characterizes the local tilt of a surface node relative to its own radial reference direction. Specifically, the direction from the point closest to the geometric center of the surface node on the cocoon's main axis to that geometric center is taken as the radial reference direction of the surface node; the local average normal vector is determined based on the 3D point cloud within the node, and the local average normal vector is uniformly oriented towards the side away from the cocoon's main axis; the angle between the local average normal vector and the node's radial reference direction is taken as the normal tilt angle. Thus, the local tilt state of two circumferentially opposed surface nodes is evaluated relative to their respective radial reference directions, instead of directly comparing two globally normal vectors with approximately opposite directions. The absolute difference in the normal tilt angle between a candidate defect-free node and a defective node is the normal tilt angle difference.

[0046] The local surface area is the sum of the areas of each triangular facet after the 3D point cloud constituting the surface node is locally triangulated. The difference in local surface area is used to evaluate the consistency of the actual coverage of two nodes. The geodesic distance distribution of adjacent nodes is a set of data consisting of the shortest surface path lengths from the geometric center of the current surface node along the reconstructed cocoon surface to the geometric centers of each first-order adjacent node. When the number of first-order adjacent nodes of two nodes is the same, the two sets of geodesic distances are arranged in ascending order and compared item by item; when the number of first-order adjacent nodes is different, the two sets of ascending geodesic distances are interpolated according to the same equiquant positions, resampled into a sequence of the same length, and then the average absolute difference of the distances at the corresponding positions is calculated as the difference in the geodesic distance distribution of adjacent nodes. This avoids the difference in the number of comparison objects caused by the surface node being at a local boundary or having invalid adjacent nodes.

[0047] When the number of 3D points used to estimate the normal vector or construct the triangular mesh in a surface node is less than the preset minimum number of points, the 3D points of the surface node and its first-level adjacent nodes can be merged to perform normal vector estimation and local surface reconstruction. Nodes that still cannot meet the minimum number of points after merging are not used as self-comparison candidate nodes, thereby ensuring that the normal tilt angle and local surface area have a stable calculation basis.

[0048] To avoid bias caused by directly adding the differences in normal dip angle, local surface area, and geodesic distance distribution between adjacent nodes due to their different dimensions, the values ​​can be normalized to the range of zero to one based on the minimum and maximum values ​​in the self-control candidate node set, and then the three normalized results can be summed. The smaller the sum, the closer the candidate defect-free node is to the defective node in terms of local three-dimensional morphology and neighborhood structure. For example, if the three normalized differences of a candidate defect-free node are 0.12, 0.08, and 0.15, with a sum of 0.35; and the differences of another candidate defect-free node are 0.20, 0.10, and 0.18, with a sum of 0.48, then the defect-free node with a sum of 0.35 is selected as the self-control node, and the defective node identifier and the self-control node identifier are associated to form a node pairing relationship.

[0049] Based on the node pairing relationship, local tangent planes are established at the defective node and the self-control node, respectively. The local tangent plane is determined by the local average normal vector of the corresponding node. The geometric center of the node is used as the origin of the local coordinate system. The direction obtained by projecting the cocoon's principal axis onto the local tangent plane is taken as the first coordinate direction, and the direction perpendicular to the first coordinate direction within the local tangent plane is taken as the second coordinate direction. The local areas covered by the two nodes are parameterized and normalized to the same coordinate range, for example, both coordinate directions are normalized to -1 to 1. Since geometric consistency screening was performed before node pairing using curvature, normal dip angle, local surface area, and geodesic distance between adjacent nodes, the local morphological differences between the two nodes are limited to a preset range. The same sampling grid is set in the unified local coordinate system, and the actual sampling position corresponding to each grid position is determined by local interpolation, so that the sampling positions of the two nodes with the same normalized coordinates form a one-to-one correspondence, thereby generating a bidirectional equipotential mapping table. The term "bidirectional" means that the corresponding position of the self-control node can be queried from the sampling position of the defective node, and vice versa.

[0050] Subsequently, equipotential registration of the three modalities was performed based on a two-way equipotential mapping table. Visible light images were resampled to a fixed grid with unified local coordinates; near-infrared spectra were mapped to the same grid according to spot size or scan position; and 3D point clouds were transformed to unified local coordinates and resampled according to the same grid or a fixed number of samples. During counterfactual evidence transfer, only one modality was selected each time, and the data at corresponding positions of the defective node and the self-control node were swapped according to the two-way equipotential mapping table. The data, node identifiers, and adjacency relationships in the cocoon topology training graph for the other two modalities remained unchanged, resulting in visible light counterfactual evidence transfer samples, near-infrared spectral counterfactual evidence transfer samples, and 3D point cloud counterfactual evidence transfer samples, respectively.

[0051] During data exchange, the node to which the data belonged before the exchange, its local coordinate position, and the node to which it belonged after the exchange, as well as its modality type, are recorded. A reverse reset mapping table is then generated in the opposite direction to the exchange direction. Finally, the three types of counterfactual evidence migration samples formed by the same defective node are associated with the node pairing relationship, the bidirectional equiposition mapping table, and the reverse reset mapping table and stored together to form a counterfactual evidence migration sample group. This provides a consistent data foundation for determining the predicted decrease of the defective node, the predicted increase of the self-control node, the defective location migration, and the reset residual.

[0052] Step S103: Use the silkworm cocoon topology training graph to perform the first stage training of the multimodal large model, update the model parameters while keeping the model structure unchanged, and obtain the first stage model; input the counterfactual evidence transfer sample group into the first stage model, generate the defect mode transfer vector according to the predicted descent of defect nodes, the predicted rise of self-control nodes and the defect position transfer, and form the defect mode transfer prototype according to the defect type.

[0053] Step S103 is used to enable the multimodal large model to first learn the basic discriminant relationship between cocoon quality and defects, and then use the counterfactual evidence transfer sample group obtained in step S102 to extract the response patterns of different defects to different modal evidence. The multimodal large model is a neural network model capable of simultaneously receiving visible light images, near-infrared spectra, and 3D point cloud information, and jointly representing and inferring different modal information. In one embodiment, the multimodal large model includes a visible light feature encoding module, a near-infrared spectral feature encoding module, a 3D morphological feature encoding module, a node feature fusion module, a topology relationship aggregation module, and a prediction output module. The three feature encoding modules extract the three types of modal features corresponding to each surface node, the node feature fusion module fuses the three types of features belonging to the same surface node, the topology relationship aggregation module transfers and aggregates information based on the adjacency relationship between surface nodes, and the prediction output module outputs the quality grade, defect type, and the predicted value of each surface node for each defect type. The predicted value represents the probability that the corresponding surface node has a specified defect type, and can be represented by a probability value between 0 and 1.

[0054] In the first stage of training, the silkworm cocoon topology training atlas is input into the multimodal large model. The quality grade, defect type, and defect region labeled in step S101 are used as supervision information to make the predicted quality grade, defect type, and defect region approximate the corresponding labels. Cross-entropy loss, binary cross-entropy loss, or other classification and localization losses can be used for training. During the first stage of training, the model modules and their connections remain unchanged. The internal weights and bias parameters of the model are updated through backpropagation until the classification errors of the training set and validation set tend to stabilize, thus obtaining the first-stage model. Keeping the model structure unchanged means that no network modules are added, deleted, or reconnected during training, and the model parameters are not restricted from being updated during training.

[0055] After obtaining the first-stage model, the counterfactual evidence transfer sample groups are input into the first-stage model and compared with the prediction results of the corresponding pre-swap samples. The decrease in defect node prediction represents the degree of decrease in the predicted value of the defect node for the original defect type after the defect node loses evidence of a certain modality defect. It can be obtained by subtracting the predicted value of the defect node after the swap from the predicted value of the defect node before the swap. For example, if the predicted value of a macular defect node before the swap is 0.84, and it becomes 0.31 after the visible light modality swap, then the corresponding decrease in defect node prediction is 0.53.

[0056] The increase in prediction from the control node represents the degree of increase in the predicted value for the same defect type after the control node receives evidence of the same modal defect. For example, if the macular prediction value before the control node swap is 0.08, and it becomes 0.67 after receiving visible light data from the defective node, then the increase in prediction from the control node is 0.59. Therefore, a single modal swap can simultaneously reflect both the decrease in response of the defective node after losing defect evidence and the increase in response after the control node receives defect evidence.

[0057] The defect location migration amount is used to characterize the degree to which the main defect location predicted by the model migrates to the self-control node after modal evidence exchange. For a specified defect type, the surface node with the highest predicted value can be determined as the predicted defect location, or the predicted defect location can be determined by weighting multiple surface nodes with predicted values ​​higher than a threshold according to their predicted values. The distance between nodes is determined according to the axial and circumferential measurable positional relationship established in step S101, where the circumferential distance is the shortest interval distance along the circumferential direction.

[0058] Specifically, the distances from the predicted defect location to the self-control node before and after the swap are first determined. The difference between these two distances is then divided by the distance from the defect node to the self-control node, and the result is taken as the defect location migration amount. The defect location migration amount is positive when the predicted defect location is closer to the self-control node after the swap; zero when the distance remains unchanged before and after the swap; and negative when the predicted defect location is farther from the self-control node after the swap. Therefore, the defect location migration amount does not reflect the absolute distance of arbitrary movement of the predicted location, but rather the degree to which the predicted defect location approaches the self-control node along the defect evidence migration direction.

[0059] For example, a defect node is located in the ninth axial interval and the third circumferential interval, while the self-control node is also located in the ninth axial interval and the ninth circumferential interval. Before the visible light data swap, the highest predicted value of the macula is located at the defect node, and the shortest circumferential distance from the predicted defect location to the self-control node is six intervals. After the swap, the highest predicted value moves to the ninth axial interval and the eighth circumferential interval, at which point the distance from the predicted defect location to the self-control node is one interval. Since the distance from the defect node to the self-control node is six intervals, the defect location migration is the difference between six intervals and one interval, divided by six intervals, which is approximately 0.83. This indicates that the predicted defect location has clearly migrated towards the self-control node where visible light defect evidence was obtained.

[0060] For each interchanged mode, the predicted decrease of the corresponding defect node, the predicted increase of the self-control node, and the defect location migration are combined in a fixed order, and the results obtained from the three modes are arranged in the same order to form the defect mode migration vector of the counterfactual evidence migration sample group. To reduce the influence of different value ranges, each component can be normalized to the same numerical range according to the value range of the training set.

[0061] Finally, defect mode transfer vectors of the same defect type are grouped together. After removing obviously abnormal samples, the mean or median is calculated for each vector dimension to form a defect mode transfer prototype for the corresponding defect type. The defect mode transfer prototype represents the representative multimodal evidence transfer response pattern of this type of defect in the overall training samples. Different defect types generate corresponding defect mode transfer prototypes, which are saved together with the counterfactual evidence transfer sample group to provide supervision for the second-stage training of the first-stage model in step S104.

[0062] Furthermore, the first-stage training of the multimodal large model using the silkworm cocoon topology training graph set, updating the model parameters while maintaining the model structure, yields the first-stage model; the counterfactual evidence transfer sample group is input into the first-stage model, and a defect mode transfer vector is generated based on the predicted descent of defective nodes, the predicted rise of self-control nodes, and the defect location transfer amount, and a defect mode transfer prototype is formed according to the defect type, including: The first-stage model is obtained by training a multimodal large model with quality grade, defect type and defect area label. The samples before the swap are input into the first-stage model, and a baseline defect response map is generated based on the predicted values ​​of each surface node for the corresponding defect type. Each counterfactual evidence migration sample is input into the first-stage model to generate a migration defect response map. Based on the baseline defect response map and the migration defect response map, the predicted decrease of defect nodes, the predicted increase of self-control nodes, and the migration of defect locations are determined, and a single-modal migration record is generated. Based on the silkworm cocoon topology training graph, the shortest surface node path from the defect node to the self-control node is determined. The path deviation of the predicted response center from the shortest surface node path is calculated, and the proportion of the predicted increment of the outer surface node to the total predicted increment is calculated as the bypass leakage amount. The difference between the predicted decrease of the defect node and the predicted increase of the self-control node is calculated based on the single-mode migration record as the migration conservation bias. Single-mode migration records in which the predicted decrease of the defect node, the predicted increase of the self-control node, and the migration of the defect location are all positive, and the migration conservation bias, path bias, and bypass leakage are all lower than the corresponding thresholds are marked as valid migration records. Following a fixed order of visible light, near-infrared, and 3D point clouds, the predicted descent of defect nodes, the predicted ascent of self-control nodes, and the defect location migration in the effective migration records are arranged to form a defect mode migration vector. The median of the corresponding components of the effective modes is then taken according to the defect type to form a defect mode migration prototype.

[0063] This implementation further defines the specific process of the first-stage model forming the defect mode migration vector and the defect mode migration prototype, so that the model's response to defect evidence is not only reflected in the change of prediction intensity, but also in the spatial migration law of defect evidence between nodes on the surface of the cocoon.

[0064] First, using the silkworm cocoon topology training map obtained in step S101 as training data, the quality grade, defect type, and defect region labels are input as supervisory information into the multimodal large model for the first stage of training. During training, the network structure of the multimodal large model and the connection relationships between its functional modules remain unchanged; only the model parameters are updated so that the model can output the quality grade, defect type, and predicted values ​​of each surface node for different defect types for the whole silkworm cocoon. Once the classification errors of the training and validation sets stabilize, the first-stage model is obtained.

[0065] For the pre-exchange samples corresponding to the counterfactual evidence transfer sample group generated in step S102, these samples are input into the first-stage model. The predicted value of each surface node for the labeled defect type of the sample is read, and the surface nodes are arranged according to their axial and circumferential positions in the cocoon topology training map to form a baseline defect response map. The baseline defect response map is used to describe the predicted response distribution of a specified defect type on the surface of the whole cocoon without modal evidence transfer, and the predicted value of the surface node is stored at each node position.

[0066] Counterfactual evidence migration samples corresponding to visible light, near-infrared spectroscopy, and 3D point clouds are input into the same first-stage model, forming a migration defect response map with the same node arrangement as the baseline defect response map. The baseline defect response map and the migration defect response map are compared node by node, and the predicted decrease in defect node, the predicted increase in self-control node, and the defect location migration are determined according to the method defined in step S103. For each interchanged mode, the interchanged mode type, the predicted decrease in defect node, the predicted increase in self-control node, the defect location migration, and the corresponding baseline and migration defect response maps are associated and saved, forming a single-mode migration record. Thus, a set of counterfactual evidence migration samples typically generates single-mode migration records corresponding to visible light, near-infrared spectroscopy, and 3D point clouds, respectively.

[0067] To further determine whether the predicted response has shifted in an orderly manner according to the spatial direction of the transferred evidence, the shortest surface node path from the defective node to the self-control node is determined based on the cocoon topology training graph. The shortest surface node path is the sequence of nodes with the smallest cumulative surface distance from the defective node to the self-control node, using surface nodes as vertices and topological connections between adjacent surface nodes as edges. The edge length can be taken as the geodesic distance between the geometric centers of two adjacent surface nodes along the reconstructed cocoon surface. This node sequence can be obtained using Dijkstra's algorithm or other existing graph search methods capable of determining the shortest path in the graph.

[0068] The predicted response center is used to characterize the comprehensive response location of a certain defect type on the surface of a silkworm cocoon. The predicted values ​​of each surface node for the target defect type can be used as weights to weight the axial and circumferential positions of the surface nodes to obtain the predicted response center. To avoid significant interference from nodes with low predicted values, only surface nodes with predicted values ​​higher than a preset response threshold can be used for calculation. The path deviation represents the degree to which the predicted response center in the migration defect response map deviates from the shortest surface node path itself. In practice, the path node closest to the predicted response center is determined in the shortest surface node path, and the surface distance from the predicted response center to this path node is used as the path deviation. This deviation can also be further normalized by dividing by the shortest surface distance from the defect node to the self-control node. The smaller the path deviation, the closer the predicted response center is to the expected migration path between the defect node and the self-control node.

[0069] For example, if the shortest surface distance from the defect node to the self-control node is 18 mm, and the predicted response center is 1.8 mm from the shortest surface node after evidence swap, the normalized path bias is 0.1; if the predicted response center is 7.2 mm from the shortest surface node after another mode swap, the normalized path bias is 0.4, indicating that the response caused by the latter mode deviates more from the expected migration path.

[0070] To evaluate the spatial diffusion of the predicted response around the shortest surface node path, a migration channel region is further established based on the shortest surface node path. The migration channel region includes all path nodes in the shortest surface node path and first-order adjacent nodes directly topologically connected to any path node. Thus, the shortest surface node path represents the central path of defect evidence migrating from the defect node to the self-control node, used to determine path deviation; the migration channel region is a strip-shaped area formed by the extension of the shortest surface node path to surrounding first-order adjacent nodes, used to determine bypass leakage. The two spatial ranges are distinct from each other.

[0071] Bypass leakage is used to evaluate whether a large number of the newly added defect prediction responses after modal evidence exchange occur outside the migration channel region. Specifically, the predicted value of each surface node in the migration defect response map is subtracted from the predicted value of the same surface node in the baseline defect response map, and only the portion greater than 0 is retained as the prediction increment of that node. Surface nodes located outside the migration channel region are designated as path-outside surface nodes. The prediction increments of path-outside surface nodes and the prediction increments of all surface nodes in the entire cocoon are accumulated separately, and the ratio of the former to the latter is taken as the bypass leakage. When all prediction increments are 0, it indicates that the evidence exchange did not form an identifiable positive response, and the single-modal migration record is not considered a valid migration record.

[0072] For example, if the sum of all positive prediction increments generated by a certain visible light counterfactual evidence migration sample is 2.0, and the sum of prediction increments of path outer surface nodes outside the migration channel region is 0.2, then the bypass leakage is 0.1; if the total prediction increment of another sample is 1.8, and the prediction increment of path outer surface nodes reaches 0.9, then the bypass leakage is 0.5, indicating that there is a significant non-target region diffusion in the model response.

[0073] The migration conservation bias is used to evaluate whether there is a correlation between the prediction decrease caused by the loss of evidence of a specified modality defect at a defective node and the prediction increase caused by the acquisition of evidence of the same modality defect at a control node. Specifically, the absolute difference between the prediction decrease at the defective node and the prediction increase at the control node can be taken as the migration conservation bias. The closer the two values ​​are, the smaller the migration conservation bias, indicating that the model's reduced defect response at the defective node is more consistent with the increased defect response at the control node. For example, if the prediction decrease at the defective node is 0.56 and the prediction increase at the control node is 0.52, then the migration conservation bias is 0.04.

[0074] For each single-modal migration record, a migration record is marked as valid only if the predicted decrease of the defect node is greater than 0, the predicted increase of the self-control node is greater than 0, the migration of the defect location is greater than 0, and the migration conservation bias, path bias, and bypass leakage are all below their respective thresholds. Each threshold can be predetermined based on the statistical distribution of correctly migrated samples in the independent validation set and remains unchanged during the same training process. For example, the high quantile values ​​of the indicators corresponding to correctly migrated samples in the validation set can be used as the migration conservation bias threshold, path bias threshold, and bypass leakage threshold.

[0075] After valid migration records are generated, defect mode migration vectors are formed by arranging the predicted decrease of defect nodes, the predicted increase of self-control nodes, and the migration amount of defect positions in each valid migration record according to a fixed order of visible light, near-infrared spectroscopy, and 3D point cloud. If a single-mode migration record corresponding to a certain mode does not meet the valid migration conditions, that mode is not included in the valid mode statistics for the corresponding defect type. All defect mode migration vectors are collected according to defect type, and the median values ​​for the same defect type, the same valid mode, and the same vector position are taken to obtain the defect mode migration prototype corresponding to that defect type. Using the median can reduce the impact of a small number of outlier samples on the representative migration patterns.

[0076] Therefore, for different defect types, defect mode migration prototypes are obtained, characterized by the predicted decrease of defect nodes in the effective mode, the predicted increase of self-control nodes, and the defect location migration. These defect mode migration prototypes not only reflect the prediction contribution of different modes to specific cocoon defects, but also exclude migration results with incorrect response directions, spatial path deviations, and significant response leakage outside the migration channel region, thus providing a stable and spatially constrained target response pattern for the subsequent second-stage training.

[0077] Step S104: Use the silkworm cocoon topology training graph, the counterfactual evidence transfer sample group, and the defect mode transfer prototype to train the first-stage model in the second stage. While keeping the model structure unchanged, continue to update the model parameters so that the defect mode transfer vector approximates the defect mode transfer prototype, and obtain the second-stage model.

[0078] Step S104, building upon the first-stage model's capabilities in cocoon quality classification, defect type identification, and defect region localization, further utilizes the multimodal evidence response of the defect mode transfer prototype constraint model obtained in step S103. This enables the model to establish stable evidence transfer patterns for the same defect type. The second-stage training continues to use the first-stage model obtained in step S103, without rebuilding the network or adding or deleting the visible light feature encoding module, near-infrared spectral feature encoding module, 3D morphological feature encoding module, node feature fusion module, topological relationship aggregation module, or prediction output module. It only updates the existing weights and bias parameters of the first-stage model.

[0079] During the second stage of training, the cocoon topology training map and the corresponding counterfactual evidence transfer sample set are input into the first stage model. For the visible light mode interchange samples, near-infrared spectral mode interchange samples, and 3D point cloud mode interchange samples in the counterfactual evidence transfer sample set, the defect node prediction decrease, self-control node prediction increase, and defect position transfer amount corresponding to the current model are obtained in the same way as in step S103, and the defect mode transfer vector is formed according to the fixed order determined in step S103.

[0080] The defect mode transfer prototype formed in step S103 serves as a fixed target response pattern during the second-stage training process. Its value remains unchanged during the second-stage training and is not recalculated with updates to the model parameters from the first stage. For each counterfactual evidence transfer sample group, a defect mode transfer prototype of the same defect type is selected based on its defect type. The defect mode transfer vector generated by the current model is compared component by component with the defect mode transfer prototype. The phrase "making the defect mode transfer vector approximate the defect mode transfer prototype" means that through model parameter updates, the predicted decrease in defect node values, the predicted increase in self-control node values, and the defect location migration values ​​corresponding to samples of the same defect type gradually approach the representative values ​​of that defect type, without requiring all components to be completely equal.

[0081] In practice, the absolute or squared difference between each component of the defect mode migration vector and the corresponding component of the defect mode migration prototype can be calculated. Then, the average of these differences is calculated to obtain the prototype bias. For example, in the defect mode migration prototype corresponding to a certain macular defect, the predicted decrease in defect node, the predicted increase in self-reference node, and the defect position migration in the visible light mode are 0.62, 0.68, and 0.74, respectively. The corresponding values ​​obtained by the current model are 0.45, 0.50, and 0.55. Through the second stage of training, these values ​​can be gradually brought closer to 0.62, 0.68, and 0.74. Near-infrared spectral modes and 3D point cloud modes are processed in the same way.

[0082] To ensure that the defect location migration can participate in the backpropagation of model parameters, the weighted method described in step S103 for determining the predicted defect location is preferred in the second stage of training. Specifically, the spatial position of each surface node can be normalized and weighted according to its predicted value for the target defect type to obtain a continuously changing predicted defect location, and then the defect location migration can be calculated based on this. Since the predicted defect location changes continuously with the predicted values ​​of each surface node, the prototype bias generated by the defect location migration can change with the model output and participate in the model parameter update through backpropagation. Using the node with the highest predicted value to determine the predicted defect location can be used for model inference or result evaluation, but the weighted location method described above is preferred in the second stage of training.

[0083] The second-stage training aims to reduce the prototype bias between the defect mode transfer vector and the corresponding defect mode transfer prototype. To avoid weakening the quality classification and defect identification capabilities established in the first stage during model parameter updates, in a preferred embodiment, the classification loss and defect region localization loss used in the first stage can be retained as auxiliary training constraints and participate in parameter updates together with the prototype bias. For example, the weights of the classification loss and defect region localization loss can be set to 1, and the prototype bias weight can be set to 0.5. These auxiliary training constraints are used to maintain the existing classification and localization capabilities of the first-stage model and are not necessary constraints for forming the defect mode transfer prototype.

[0084] In each round of the second-stage training, the current defect mode transfer vector and prototype bias are calculated using the counterfactual evidence transfer sample set, and the parameters of the first-stage model are updated through backpropagation. When auxiliary training constraints are used, the cocoon topology training map can be input simultaneously to calculate the classification loss and defect region localization loss, and the parameters are updated together. When the average prototype bias of the validation set changes less than a preset range or falls below a preset threshold for several consecutive training rounds, the second-stage training can be stopped, and the second-stage model is obtained. The second-stage model has the same model structure as the first-stage model. The difference is that the model parameters are further updated by the defect mode transfer prototype constraint, which provides a basis for evaluating the predictive stability of the model after evidence reset in step S105.

[0085] Furthermore, the second-stage training of the first-stage model using a silkworm cocoon topology training graph, a counterfactual evidence transfer sample set, and a defective mode transfer prototype, while maintaining the model structure, continues to update the model parameters to make the defective mode transfer vector approximate the defective mode transfer prototype, thus obtaining the second-stage model, including: Input the counterfactual evidence transfer sample group into the first-stage model to generate defect mode transfer vectors, and arrange the node prediction values ​​of the axial interval where the defect node is located according to the circumferential interval number to generate a circumferential response sequence. The circumferential response sequences before and after the interchange are cyclically shifted and matched. The circumferential phase migration is generated by the interval displacement when the correlation value is the maximum. The prototype target displacement is generated by multiplying the circumferential interval from the defect node to the self-control node by the defect position migration in the defect mode migration prototype. The axial interval is divided into two semi-circular node chains with the defect node and the self-control node as endpoints. The positive prediction increments are accumulated respectively. The circumferential closure deviation record is generated by the difference between the two accumulated values ​​and the proportion of the positive prediction increment outside the axial interval to the total positive prediction increment. The second-stage training loss is generated by combining the component differences between the defect mode transfer vector and the defect mode transfer prototype, the displacement difference between the circumferential phase transfer amount and the prototype target displacement, and the circumferential closure deviation. The parameters of the first-stage model are updated based on the training loss of the second stage, and the defect mode transfer vector is regenerated until the component difference, displacement difference and circumferential closure deviation record are lower than the threshold, thus obtaining the second-stage model.

[0086] This implementation method is used in the second stage of training to further utilize the closed circumferential topological relationship of the cocoon surface to spatially constrain the predicted response generated by the counterfactual evidence migration, so that the model not only approaches the defect mode migration prototype in terms of the numerical value of the defect mode migration vector, but also makes the overall movement direction and distribution range of the defect response in the circumferential direction of the cocoon conform to the spatial relationship of the defect node migrating to the self-control node.

[0087] The counterfactual evidence transfer sample group is input into the first-stage model to obtain the defect mode transfer vector as described above. Simultaneously, the axial interval containing the defect node is selected, and all surface nodes within this interval are arranged sequentially according to their circumferential interval numbers. The predicted values ​​for each surface node for the current defect type are then read, forming a circumferential response sequence. This circumferential response sequence is contiguous and represents the distribution of the defect prediction response around the cocoon's main axis at the same axial position. For example, when the circumference is divided into 12 intervals, the circumferential response sequence contains 12 predicted values ​​arranged according to the 1st to 12th circumferential intervals.

[0088] Based on the pre-modal interchange samples and the counterfactual evidence transfer samples, pre-interchange and post-interchange circumferential response sequences are formed, and cyclic shift matching is performed. Cyclic shift refers to keeping the pre-interchange circumferential response sequence unchanged and shifting the post-interchange circumferential response sequence cyclically along the circumferential direction, interval by interval. For each interval shift, the correlation between the predicted values ​​at corresponding positions of the two sequences is calculated. The predicted value at the end of the shift is reconnected to the beginning of the sequence, thus not disrupting the circumferential closure relationship of the cocoon. The correlation can be determined using the normalized correlation coefficient; a larger correlation value indicates that the two response distributions are closer at that shift. The interval shift of the post-interchange circumferential response sequence relative to the pre-interchange circumferential response sequence when the correlation value is maximum is determined as the circumferential phase shift, and its sign is determined according to the circumferential direction towards the control node.

[0089] For example, the defect node is located in the 3rd circumferential interval, and the self-control node is located in the 9th circumferential interval, with a circumferential interval interval between them. After swapping the circumferential response sequence, a cyclic shift is performed. The maximum correlation value is obtained when the sequence moves 5 intervals along the direction from the defect node to the self-control node. In this case, the circumferential phase shift is 5 intervals, indicating that the overall defect response distribution has clearly shifted towards the direction of the self-control node.

[0090] The defect location migration amount in the defect mode transfer prototype represents the representative spatial migration degree of the same defect type when migration occurs in the corresponding modal evidence. The prototype target displacement is obtained by multiplying the circumferential interval from the defect node to the self-control node by the defect location migration amount. For example, if the circumferential interval is 6 intervals, and the defect location migration amount in the defect mode transfer prototype is 0.8, then the prototype target displacement is 4.8 intervals. In actual training, continuous displacement values ​​can be retained, or they can be converted to the closest interval displacement based on the discretization accuracy of the circumferential nodes. The absolute difference between the circumferential phase migration amount and the prototype target displacement constitutes the displacement difference, which is used to evaluate whether the overall circumferential response migration generated by the model conforms to the typical migration degree of this defect type.

[0091] Since the defect node and the self-control node are circumferentially opposite each other, two semi-circular node chains are formed on the closed node loop within the same axial interval, each formed by the two endpoints along two circumferential directions. Each semi-circular node chain contains a set of continuous surface nodes leading from the defect node to the self-control node. For identical surface nodes before and after the swap, the predicted value after the swap is subtracted from the predicted value before the swap, and the portion greater than 0 is determined as the positive prediction increment. The positive prediction increments of each node within the two semi-circular node chains are accumulated separately, and the absolute difference between the two accumulated results is used to evaluate whether the response to the newly added defect is excessively biased towards one of the semi-circular node chains.

[0092] Simultaneously, the positive predicted increments of all surface nodes outside the axial interval where the defective node is located are statistically analyzed and divided by the positive predicted increments of all surface nodes in the whole cocoon to obtain the response ratio outside the axial interval. When all positive predicted increments are 0, the migration is considered to have not generated a valid positive response and is not used to form a normal circumferential closure constraint. The cumulative difference of the positive predicted increments of the two semi-circular node chains and the response ratio outside the axial interval are jointly saved to form a circumferential closure deviation record. For example, if the positive predicted increments of the two semi-circular node chains are 1.10 and 0.96 respectively, the cumulative difference is 0.14; if the total positive predicted increment is 2.50, of which the positive predicted increment outside the axial interval where the defective node is located is 0.20, the response ratio outside the axial interval is 0.08. The smaller the two values, the more concentrated the response is in the circumferential closed region where the defective node is located, and there is no obvious unilateral abnormal diffusion between the two semi-circular paths reachable from the control node.

[0093] In the second stage of training, the current defect mode transfer vector is compared component by component with the defect mode transfer prototype of the same defect type to obtain the component difference; the circumferential phase transfer amount is compared with the prototype target displacement to obtain the displacement difference; and the cumulative difference and the proportion of the response outside the axial interval are read from the circumferential closure deviation record. To eliminate the influence of different index value ranges, the above indices can be normalized to the range of 0 to 1, and then weighted and summed according to preset weights to obtain the second stage training loss. The component difference between the defect mode transfer vector and the defect mode transfer prototype is used to maintain the consistency of the prototype response, the displacement difference is used to constrain the overall circumferential response transfer degree, and the circumferential closure deviation record is used to limit the abnormal diffusion of the predicted response into the irrelevant axial region or the unilateral half-circumferential region.

[0094] Backpropagation is performed based on the training loss from the second stage, and the model parameters are updated while maintaining the network structure of the first-stage model. After each training round, the counterfactual evidence transfer sample group is re-inputted, and the defect mode transfer vector, circumferential phase transfer amount, and circumferential closure deviation record are recalculated. Training stops when the component difference, displacement difference, and circumferential closure deviation record corresponding to each effective mode in the validation set are all below the corresponding threshold for multiple consecutive training rounds, or when their changes are below the preset convergence range, resulting in the second-stage model. This allows the second-stage model to learn the defect evidence transfer rules consistent with the closed circumferential structure of the cocoon, providing a stable model foundation for subsequent resetting training.

[0095] Step S105: Reset the counterfactual evidence transfer sample group, calculate the prediction difference of the second-stage model before and after reset as the reset residual, weight the counterfactual evidence transfer sample group whose reset residual exceeds the threshold to train the second-stage model, and continue to update the model parameters until the classification error and reset residual converge, thus obtaining the cocoon sorting decision model.

[0096] Step S105 is used to perform restorative verification on the predicted changes formed by the model during the counterfactual evidence transfer training process after obtaining the second-stage model in step S104, and to further adjust the model parameters for training samples with insufficient restorative ability, so that the model can not only respond to the transfer of valid defective evidence, but also restore the corresponding predicted state after the defective evidence restores its original spatial attribution.

[0097] The reset refers to restoring the swapped modal data in the counterfactual evidence migration sample group to their original surface nodes according to the defect node identifier, self-control node identifier, and swapped modal type saved in step S102. For example, in the visible light modal swap sample, if the visible light data of the defect node is migrated to the self-control node, and the visible light data of the self-control node is migrated to the defect node, then during reset, the two sets of visible light data are restored to the original defect node and the original self-control node, respectively, while the near-infrared spectral and 3D point cloud data remain in their original states. The near-infrared spectral modality and the 3D point cloud modality are reset in the same way. The data content and node affiliation after reset remain consistent with those before the modal swap.

[0098] To ensure that the reset residual accurately reflects the predictive recovery capability of the model after parameter training, before each round of further training, the current second-stage model is used to infer the samples before the swap, and the predicted quality level, defect type, and defect node values ​​are saved as baseline predictions. Then, the parameters are updated using the corresponding counterfactual evidence transfer sample group, the counterfactual evidence transfer sample group is reset, and the parameter-updated second-stage model is used to infer the reset samples. The reset residual is the difference between the reset prediction result and the baseline prediction saved before the parameter update. Therefore, it does not compare the difference obtained by performing two inferences on the exact same input using a fixed model, but rather evaluates whether the model can still maintain its original judgment on the cocoon restored to its original evidence state after counterfactual evidence transfer training.

[0099] The reset residual can be determined by at least one of the following: the prediction difference of quality grade, the prediction difference of defect type, and the prediction difference of defect node. Preferably, the absolute difference between the corresponding predicted values ​​is calculated after normalizing each predicted value to the same range, and then the average value is taken as the reset residual for the counterfactual evidence transfer sample group. For example, before a round of parameter updates, the second-stage model predicts a yellow spot value of 0.86 for the original sample of a certain yellow-spotted silkworm cocoon; after training with the counterfactual evidence transfer sample of this cocoon, the sample is reset, and the updated model predicts a yellow spot value of 0.72 for the same defect node, then the prediction difference is 0.14. If the average difference between the predictions of quality grade and other defect-related factors results in a reset residual of 0.11 for this sample group, and the preset reset residual threshold is 0.08, then this counterfactual evidence transfer sample group belongs to the sample group whose reset residual exceeds the threshold.

[0100] The reset residual threshold is used to distinguish between samples whose predictions remain stable after counterfactual evidence transfer training and samples with large prediction biases. It can be predetermined based on the statistical results of the reset residuals of each counterfactual evidence transfer sample group in the validation set. For example, the high quantile of the reset residuals of normally converged samples in the validation set can be used as the threshold, or a fixed threshold can be determined based on the allowable fluctuation range of quality classification predictions. A uniform determination method is used during training to avoid arbitrarily changing the threshold for a single sample.

[0101] For counterfactual evidence transfer sample groups whose reset residuals exceed a threshold, the training weight of these sample groups in subsequent training is increased. The training weight represents the degree to which the training error generated by this sample group affects the model parameter updates. For example, the training weight of a normal sample group is set to 1, while the weight of a sample group with reset residuals exceeding the threshold can be increased to 1.5. During weighted training, both the counterfactual evidence transfer state and the reset state of the sample group are used simultaneously, allowing the model to maintain the defective mode transfer response formed in step S104 and strengthening the prediction retention constraint in the reset state. This avoids excessive drift in the model's classification results of the original cocoons due to simply strengthening the evidence transfer response. Specific weights can be set in stages according to the degree to which the reset residuals exceed the threshold, but a predetermined weighting rule is used within the same training batch.

[0102] During further training, the network structure of the second-stage model remains unchanged; only the weights and bias parameters are updated. The classification error is determined by the difference between the quality grade, defect type, and other labels in the cocoon topology training graph and the model's predictions, ensuring the model continues to possess correct cocoon sorting judgment capabilities. The reset residual is used to evaluate the model's ability to recover its original predictive state after counterfactual evidence transfer training. After each round of weighted training, the classification error and average reset residual on the validation set are recalculated. When the changes in classification error and average reset residual over multiple consecutive training rounds are all below the preset convergence range, and the average reset residual is not higher than the allowable threshold, the classification error and reset residual are considered to have converged. For example, if the changes in classification error and average reset residual are all less than 0.001 over five consecutive training rounds, training can be stopped.

[0103] After training, a cocoon sorting decision model was obtained. This cocoon sorting decision model maintains the same network structure as the second-stage model, but its model parameters are further trained with sample weighting driven by reset residuals. This allows the model to maintain the defect mode transfer law while reducing the original sample prediction drift caused by counterfactual evidence transfer training, thereby forming a stable ability to judge cocoon quality grade, defect type, and defect area.

[0104] Furthermore, the process involves resetting the counterfactual evidence transfer sample group, calculating the prediction difference between the second-stage model before and after resetting as the reset residual, weighting the counterfactual evidence transfer sample group with reset residuals exceeding a threshold to train the second-stage model, and continuing to update the model parameters until the classification error and reset residual converge, thus obtaining the cocoon sorting decision model, including: The nodes are sorted according to the feature differences between the defect nodes and the self-control nodes at the corresponding positions of the interchange modes, and divided into multiple reset subsets to generate a reset sequence. According to the reset sequence, each reset subset is restored to the node before the interchange. After each reset subset is completed, it is input into the second stage model to record the predicted value of the defect node, the predicted value of the self-control node and the predicted defect location, and generate a multi-level reset response trajectory. Based on the multi-level reset response trajectory, the predicted recovery amount of the defect node, the predicted cancellation amount of the self-control node, and the predicted defect location return amount are determined. They are compared with the predicted decrease amount of the defect node, the predicted increase amount of the self-control node, and the defect location migration amount, respectively, to generate a migration reset hysteresis vector. The proportion of the number of adjacent reset levels in which the predicted change direction is opposite to the preset recovery direction is calculated to generate the reset monotonic violation rate. After completing the reset of all reset subsets, the reset residual is calculated. Samples whose reset residual exceeds the threshold and whose transfer reset hysteresis vector or reset monotonic violation rate exceeds the corresponding threshold are identified as reset difficult samples. The training weights are determined based on the reset residual, transfer reset hysteresis vector and reset monotonic violation rate to generate a closed-loop weighted training set. The second-stage model is trained using a closed-loop weighted training set. The model structure remains unchanged while the model parameters are updated until the classification error and reset residual converge, thus obtaining the silkworm cocoon sorting decision model.

[0105] This implementation method performs graded reset of counterfactual evidence migration samples, observes the continuous prediction changes of the second-stage model during the process of defective evidence gradually returning to its original spatial position, and selects samples with difficult reset for weighted training.

[0106] For each counterfactual evidence transfer sample in the counterfactual evidence transfer sample group, the corresponding sampling positions between the defective node and the self-control node are determined according to the aforementioned two-way equivalence mapping relationship. For the currently swapped modality, the feature difference value of each pair of corresponding sampling positions before the swap is calculated. The feature difference value is used to represent the degree of difference between the defective node data and the self-control node data at that position. For the visible light modality, the mean absolute difference can be calculated on the normalized color and texture features; for the near-infrared spectral modality, the mean absolute difference can be calculated on the normalized spectral features in a unified band; for the 3D point cloud modality, the mean absolute difference can be calculated on the morphological features after local coordinate registration. Consistent feature processing and normalization methods are used within each modality to enable comparison of feature differences at different sampling positions.

[0107] The corresponding sampling positions in the same counterfactual evidence transfer sample are sorted from largest to smallest according to their feature differences, and then divided into multiple reset subsets according to a predetermined number of identical or nearly identical subsets, with the reset subsets having larger feature differences listed first, thus forming a reset sequence. Each corresponding sampling position belongs to only one reset subset, and all reset subsets collectively cover all sampling positions that have been interchanged in the current modality. For example, when a node has 80 valid corresponding sampling positions, it can be divided into 4 reset subsets, each containing 20 corresponding sampling positions. The specific number of reset subsets can be determined based on the number of samples from the node, as long as it ensures that at least two consecutive reset levels can be formed.

[0108] The reset is performed sequentially according to the reset sequence. Upon completion of each reset subset, the data within that subset is restored to its original node and corresponding local coordinate position before the swap, based on the reverse reset mapping relationship. Data that has already been reset remains in the reset state, while unprocessed data continues to remain in the counterfactual evidence migration state. Data from the other two unswapped modalities remains unchanged. This generates multiple intermediate samples that gradually transition from a fully migrated state to a fully reset state. Each intermediate sample is input into the second-stage model, recording the predicted value of the defect node corresponding to the target defect type, the predicted value of the self-control node, and the predicted defect location. These are then associated and saved according to the reset level order, forming a multi-level reset response trajectory.

[0109] The defect node prediction recovery amount represents the extent to which the predicted value of the defect node for the original defect type increases relative to the fully migrated state as reset proceeds; the self-reference node prediction cancellation amount represents the extent to which the predicted value of the self-reference node for the original defect type decreases relative to the fully migrated state; and the predicted defect location return amount represents the extent to which the predicted defect location returns from its spatial location after evidence migration to the original defect node. The predicted defect location is determined according to the aforementioned measurable axial and circumferential positional relationships. Specifically, the distance from the predicted defect location at the current reset level to the original defect node can be compared with the corresponding distance in the fully migrated state. The decrease in distance is taken as the predicted defect location return amount and normalized according to the same spatial scale used for defect location migration.

[0110] The predicted recovery amount of the defect node after all reset subsets are completed is compared with the corresponding predicted decrease amount of the defect node; the predicted cancellation amount of the self-control node is compared with the corresponding predicted increase amount of the self-control node; and the predicted defect position return amount is compared with the corresponding defect position migration amount. The absolute difference of each group of corresponding values ​​can be taken, and arranged in a fixed order to form a migration-reset hysteresis vector. The migration-reset hysteresis vector is used to evaluate whether the model response of the same modal evidence is consistent in the two opposite directions of migration out and reset. For example, if the predicted decrease amount of the defect node, the predicted increase amount of the self-control node, and the defect position migration amount during a certain visible light modality migration are 0.58, 0.54, and 0.72, respectively, and the corresponding recovery amounts after reset are 0.51, 0.50, and 0.65, then the migration-reset hysteresis vector is 0.07, 0.04, and 0.07. The smaller each component is, the closer the migration response is to the reset response.

[0111] The multi-level reset response trajectory is also used to determine the reset monotonic violation rate. The preset recovery direction is: as the reset level increases, the predicted value of the defect node should not decrease, the predicted value of the control node should not increase, and the distance from the predicted defect location to the original defect node should not increase. Adjacent reset levels are compared sequentially; if the change direction of any recorded value is opposite to the corresponding preset recovery direction, that adjacent reset level is recorded as a violation. The number of adjacent reset levels with violations is divided by the total number of adjacent reset levels to obtain the reset monotonic violation rate. For example, if there are 4 groups of adjacent reset levels with 5 reset levels, and 1 group has a violation, then the reset monotonic violation rate is 0.25. This allows identification of model responses that, although the predicted result can eventually be recovered, exhibit reverse fluctuations during the step-by-step reset process.

[0112] After completing the reset subset, the reset residuals between the fully reset samples and the samples before the swap are calculated according to the aforementioned reset processing method. Counterfactual evidence transfer samples whose reset residuals exceed the corresponding thresholds, and at least one component in the transfer reset hysteresis vector exceeds the corresponding threshold or whose reset monotonic violation rate exceeds the corresponding threshold, are identified as reset-difficult samples. Each threshold can be predetermined based on the statistical distribution of the corresponding indicators in the independent validation set and remains unchanged during the same training process.

[0113] For reset-difficult samples, training weights are determined based on reset residuals, transfer reset hysteresis vectors, and reset monotonic violation rates. First, these metrics are normalized to the range of 0 to 1. The average value of each component of the transfer reset hysteresis vector is then taken. Finally, the normalized reset residuals, average hysteresis value, and reset monotonic violation rate are summed to obtain the difficulty level. The greater the difficulty level, the higher the assigned training weight. Resetting-difficult samples, their corresponding pre-swap samples, intermediate reset samples at each level, and their training weights are correlated to form a closed-loop weighted training set.

[0114] The second-stage model is trained using a closed-loop weighted training set, maintaining the model structure and module connections unchanged, updating only the model parameters, and retaining the quality level and defect type labels to supervise the classification results. After each training round, the classification error and reset residual are recalculated on the validation set. Training stops when the changes in classification error and average reset residual are all below a preset convergence range for multiple consecutive training rounds, and the average reset residual is not higher than an allowable threshold, thus obtaining the cocoon sorting decision model. This enables the model to not only generate predictive changes corresponding to the evidence location during defect evidence migration, but also to stably return to the original predictive state after the defect evidence is gradually recovered.

[0115] Figure 2 The diagram illustrates the changes in classification error and reset residuals with training epochs during the training process of this invention. The solid line represents the result obtained using the method of this invention, while the dashed line represents the result obtained using the comparative method. As can be seen from the classification error curve on the left, this invention can quickly reduce classification error in the early stages of training and maintain a stable decrease in subsequent training, exhibiting faster convergence speed and lower final classification error compared to the comparative method. As can be seen from the reset residual curve on the right, after employing graded reset, transfer reset hysteresis constraints, and weighted training with reset-difficult samples, the reset residuals of this invention decrease rapidly with each training epoch and enter the stable region earlier. In contrast, the reset residuals of the comparative method decrease more slowly and have a higher stability value. These results demonstrate that this invention can improve the predictive consistency and recovery stability of the model in the counterfactual evidence transfer and evidence reset processes while maintaining the cocoon classification ability, thereby enhancing the training effect and reliability of the cocoon sorting decision model.

[0116] A second embodiment of this application provides an electronic device 300, the electronic device 300 comprising: Processor 301; The memory 302 is used to store a program, which, when read and executed by the processor, executes a method for training a silkworm cocoon sorting decision model based on a multimodal large model provided in the first embodiment of this application.

[0117] The third embodiment of this application provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it executes a training method for a silkworm cocoon sorting decision model based on a multimodal large model provided in the first embodiment of this application.

[0118] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A training method for a silkworm cocoon sorting decision model based on a multimodal large model, characterized in that, include: Collect visible light images, near-infrared spectra, and 3D point clouds of silkworm cocoons, label quality grades, defect types, and defect areas, determine the main axis and waist section of the cocoon from the 3D point cloud, establish axial and circumferential surface nodes and map the three types of data, and generate a silkworm cocoon topology training map. For defective nodes, select non-defective self-control nodes that are axially corresponding, circumferentially opposite, and have a curvature difference below the threshold in the same cocoon. Exchange the data of defective nodes and self-control nodes modally to generate a counterfactual evidence transfer sample group. The first stage of training of the multimodal large model is carried out using the silkworm cocoon topology training graph. The model parameters are updated while keeping the model structure unchanged to obtain the first stage model. The counterfactual evidence transfer sample group is input into the first stage model. The defect mode transfer vector is generated based on the predicted descent of defect nodes, the predicted rise of self-control nodes, and the defect location transfer. Defect mode transfer prototypes are formed according to defect type. The first-stage model was trained using a silkworm cocoon topology training graph, a counterfactual evidence transfer sample set, and a defective mode transfer prototype. The model parameters were updated while keeping the model structure unchanged, so that the defective mode transfer vector approximates the defective mode transfer prototype, thus obtaining the second-stage model. The counterfactual evidence transfer sample group is reset, and the prediction difference of the second-stage model before and after reset is calculated as the reset residual. The counterfactual evidence transfer sample group with reset residual exceeding the threshold is weighted to train the second-stage model. The model parameters are updated until the classification error and reset residual converge, and the cocoon sorting decision model is obtained.

2. The method for training a silkworm cocoon sorting decision model based on a multimodal large model according to claim 1, characterized in that, For defective nodes, in the same silkworm cocoon, select axially corresponding, circumferentially opposite, and curvature difference-below-threshold-free self-control nodes. Then, interchange the data of the defective nodes and the self-control nodes modally to generate a counterfactual evidence transfer sample group, including: Based on the axial and circumferential interval numbers of the defective nodes, the same axial interval and the opposite circumferential interval are determined. From the intersection area, defect-free nodes with curvature differences below the threshold and where neither the node itself nor its first-level adjacent nodes are covered by the defect area are selected to generate a self-comparison candidate node set. Calculate the difference in normal dip angle, local surface area, and geodesic distance distribution between each defect-free node and defective node in the self-control candidate node set. Normalize each difference and sum them. Select the defect-free node with the smallest sum as the self-control node and generate node pairing relationships. Based on the node pairing relationship, a unified local coordinate system is established on the local tangent plane between the defect node and the self-control node to determine the one-to-one correspondence between the corresponding sampling positions and generate a bidirectional equipotential mapping table. Based on the bidirectional equipotential mapping table, visible light images, near-infrared spectra, and 3D point clouds are equipotentially registered. Each time, only one type of modal data is exchanged while keeping the other two types of modal data, node identifiers, and node topology unchanged, generating three types of counterfactual evidence transfer samples and a reverse reset mapping table. The three types of counterfactual evidence migration samples are associated with node pairing relationships, bidirectional equivalence mapping tables, and reverse reset mapping tables to generate counterfactual evidence migration sample groups.

3. The method for training a silkworm cocoon sorting decision model based on a multimodal large model according to claim 1, characterized in that, The first stage of training of the multimodal large model is performed using a silkworm cocoon topology training graph. Model parameters are updated while maintaining the model structure, resulting in the first-stage model. Counterfactual evidence transfer sample groups are input into the first-stage model. Defect mode transfer vectors are generated based on the predicted descent of defective nodes, the predicted rise of self-control nodes, and the defect location transfer. Defect mode transfer prototypes are then formed according to defect type, including: The first-stage model is obtained by training a multimodal large model with quality grade, defect type and defect area label. The samples before the swap are input into the first-stage model, and a baseline defect response map is generated based on the predicted values ​​of each surface node for the corresponding defect type. Each counterfactual evidence migration sample is input into the first-stage model to generate a migration defect response map. Based on the baseline defect response map and the migration defect response map, the predicted decrease of defect nodes, the predicted increase of self-control nodes, and the migration of defect locations are determined, and a single-modal migration record is generated. Based on the silkworm cocoon topology training graph, the shortest surface node path from the defect node to the self-control node is determined. The path deviation of the predicted response center from the shortest surface node path is calculated, and the proportion of the predicted increment of the outer surface node to the total predicted increment is calculated as the bypass leakage amount. The difference between the predicted decrease of the defect node and the predicted increase of the self-control node is calculated based on the single-mode migration record as the migration conservation bias. Single-mode migration records in which the predicted decrease of the defect node, the predicted increase of the self-control node, and the migration of the defect location are all positive, and the migration conservation bias, path bias, and bypass leakage are all lower than the corresponding thresholds are marked as valid migration records. Following a fixed order of visible light, near-infrared, and 3D point clouds, the predicted descent of defect nodes, the predicted ascent of self-control nodes, and the defect location migration in the effective migration records are arranged to form a defect mode migration vector. The median of the corresponding components of the effective modes is then taken according to the defect type to form a defect mode migration prototype.

4. The method for training a silkworm cocoon sorting decision model based on a multimodal large model according to claim 1, characterized in that, The process involves resetting the counterfactual evidence transfer sample group, calculating the prediction difference of the second-stage model before and after resetting as the reset residual, weighting the counterfactual evidence transfer sample group with reset residuals exceeding a threshold to train the second-stage model, and continuing to update the model parameters until the classification error and reset residual converge, thus obtaining the cocoon sorting decision model, including: The nodes are sorted according to the feature differences between the defect nodes and the self-control nodes at the corresponding positions of the interchange modes, and divided into multiple reset subsets to generate a reset sequence. According to the reset sequence, each reset subset is restored to the node before the interchange. After each reset subset is completed, it is input into the second stage model to record the predicted value of the defect node, the predicted value of the self-control node and the predicted defect location, and generate a multi-level reset response trajectory. Based on the multi-level reset response trajectory, the predicted recovery amount of the defect node, the predicted cancellation amount of the self-control node, and the predicted defect location return amount are determined. They are compared with the predicted decrease amount of the defect node, the predicted increase amount of the self-control node, and the defect location migration amount, respectively, to generate a migration reset hysteresis vector. The proportion of the number of adjacent reset levels in which the predicted change direction is opposite to the preset recovery direction is calculated to generate the reset monotonic violation rate. After completing the reset of all reset subsets, the reset residual is calculated. Samples whose reset residual exceeds the threshold and whose transfer reset hysteresis vector or reset monotonic violation rate exceeds the corresponding threshold are identified as reset difficult samples. The training weights are determined based on the reset residual, transfer reset hysteresis vector and reset monotonic violation rate to generate a closed-loop weighted training set. The second-stage model is trained using a closed-loop weighted training set. The model structure remains unchanged while the model parameters are updated until the classification error and reset residual converge, thus obtaining the silkworm cocoon sorting decision model.

5. The method for training a silkworm cocoon sorting decision model based on a multimodal large model according to claim 1, characterized in that, The process involves collecting visible light images, near-infrared spectra, and 3D point clouds of silkworm cocoons, labeling quality grades, defect types, and defect areas, determining the cocoon's main axis and waist cross-section from the 3D point cloud, establishing axial and circumferential surface nodes, mapping the three types of data, and generating a silkworm cocoon topology training atlas, including: The main axis of the cocoon, the cross section of the cocoon waist, the two extreme points of the cocoon end, and the main direction of the cocoon waist contour are determined by the 3D point cloud, generating a cocoon positioning reference with a unique axial direction and circumferential zero position; Based on the cocoon positioning benchmark, the axial region and the circumferential region are equally divided according to the cumulative surface area, and the boundary line is adjusted with the minimum geodesic distance deviation between adjacent nodes as a constraint to generate a set of topological nodes with equal surface area. Visible light images and near-infrared spectra are mapped to a set of topological nodes with equal surface area. The effective coverage ratio of each node for the three modes and the intersection coverage ratio are calculated. Sampling positions with an intersection coverage ratio lower than the threshold are removed to generate a cross-modal co-certification node set. The defect region is mapped to the cross-modal co-certification node set. The nodes that contain both defect and non-defect regions are subdivided along the defect boundary, and the adjacency edges are reconstructed according to the geodesic adjacency relationship of the nodes before and after subdivision, generating a defect boundary-preserving node set. By maintaining the three modalities of the node set at the defect boundary, node location, effective coverage information, quality level, defect type, defect region label, and adjacency relationship association, a cocoon topology training graph is generated.

6. The method for training a silkworm cocoon sorting decision model based on a multimodal large model according to claim 1, characterized in that, The second-stage training of the first-stage model involves using a silkworm cocoon topology training atlas, a counterfactual evidence transfer sample set, and a defective mode transfer prototype. While maintaining the model structure, the model parameters are continuously updated to make the defective mode transfer vector approximate the defective mode transfer prototype, resulting in the second-stage model, which includes: Input the counterfactual evidence transfer sample group into the first-stage model to generate defect mode transfer vectors, and arrange the node prediction values ​​of the axial interval where the defect node is located according to the circumferential interval number to generate a circumferential response sequence. The circumferential response sequences before and after the interchange are cyclically shifted and matched. The circumferential phase migration is generated by the interval displacement when the correlation value is the maximum. The prototype target displacement is generated by multiplying the circumferential interval from the defect node to the self-control node by the defect position migration in the defect mode migration prototype. The axial interval is divided into two semi-circular node chains with the defect node and the self-control node as endpoints. The positive prediction increments are accumulated respectively. The circumferential closure deviation record is generated by the difference between the two accumulated values ​​and the proportion of the positive prediction increment outside the axial interval to the total positive prediction increment. The second-stage training loss is generated by combining the component differences between the defect mode transfer vector and the defect mode transfer prototype, the displacement difference between the circumferential phase transfer amount and the prototype target displacement, and the circumferential closure deviation. The parameters of the first-stage model are updated based on the training loss of the second stage, and the defect mode transfer vector is regenerated until the component difference, displacement difference and circumferential closure deviation record are lower than the threshold, thus obtaining the second-stage model.

Images