Device detection method and model training method

Abstract

The application provides a device detection method and a model training method, relates to the fields of image processing and machine vision, and comprises the following steps: obtaining a target image corresponding to a target device; processing multi-scale features of the target image to obtain channel attention features and spatial attention features of the target image, wherein the channel attention features are used for indicating features of the target device in multiple feature channels, and the spatial attention features are used for indicating features of multiple positions of the target device; and determining a device detection result of the target device according to the channel attention features and the spatial attention features. The application can improve the accuracy of device defect detection.

Description

Technical Field

[0001] This application relates to the fields of image processing and machine vision technology, and in particular to a device detection method and a model training method. Background Technology

[0002] In modern manufacturing processes, the requirements for quality control of manufactured components are becoming increasingly stringent. The quality of components is crucial to the performance of products, therefore, defect detection of manufactured components is necessary.

[0003] In related technologies, device images are input into a trained defect detection model for defect detection. However, these defect detection methods use fixed-size feature maps during model training and can only detect devices with single structures. This results in low detection accuracy when detecting tiny or complex devices. Summary of the Invention

[0004] This application provides a device detection method and a model training method to address the problem that existing defect detection methods have low detection accuracy when detecting small devices and devices with complex structures.

[0005] In a first aspect, embodiments of this application provide a device detection method, including:

[0006] Obtain the target image corresponding to the target device;

[0007] The multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations.

[0008] The device detection result of the target device is determined based on the channel attention features and the spatial attention features.

[0009] In one possible implementation, the multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image, including:

[0010] Obtain the merged features and pooled features of the multi-scale features;

[0011] The merged features are subjected to channel attention processing to obtain channel attention features;

[0012] Spatial attention features are obtained by performing spatial attention processing on the pooling features.

[0013] In one possible implementation, the merging features include an average merging feature and a maximum merging feature; channel attention processing is applied to the merging features to obtain channel attention features, including:

[0014] The average merged features are processed by a multilayer perceptron to obtain the first merged feature;

[0015] The second merged feature is obtained by processing the maximum merged feature using the multilayer perceptron.

[0016] The first merged feature and the second merged feature are merged to obtain the first target feature;

[0017] The channel attention features are obtained by processing the first target feature using a preset activation function.

[0018] In one possible implementation, the pooling features include average pooling features and max pooling features; spatial attention processing is applied to the pooling features to obtain spatial attention features, including:

[0019] The average pooling feature and the max pooling feature are concatenated to obtain the second target feature;

[0020] The second target feature is convolved to obtain the third target feature;

[0021] The spatial attention features are obtained by processing the third target features using a preset activation function.

[0022] In one possible implementation, determining the device detection result of the target device based on the channel attention features and the spatial attention features includes:

[0023] The channel attention features and the spatial attention features are fused to obtain fused attention features;

[0024] Based on the fusion attention features, the image type of the target image is determined, and the image type includes one of the following: the front of the target device, the back of the target device, and the side of the target device;

[0025] The device detection result is determined based on the fused attention features and the image type.

[0026] In one possible implementation, the multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image, including:

[0027] The target image is subjected to multi-scale feature extraction processing using a target model to obtain the multi-scale features;

[0028] The multi-scale features are processed by the target model to obtain the channel attention features and the spatial attention features;

[0029] Based on the channel attention features and the spatial attention features, the device detection result of the target device is determined, including:

[0030] The target model is used to process the channel attention features and the spatial attention features to obtain the device detection results.

[0031] Secondly, embodiments of this application provide a model training method, including:

[0032] Acquire sample data, which includes multiple sample images of the sample device and annotation information for each sample image. The annotation information includes the type of the annotated image and the detection result of the annotated device.

[0033] The feature extraction module of the preset model is used to obtain multi-scale features of each sample image;

[0034] The multi-scale features of each sample image are processed by the attention module of the preset model to obtain the fused attention features of each sample image.

[0035] The multi-task module of the preset model processes the fusion attention features of each sample image to obtain the predicted image type and the prediction device detection result of each sample image.

[0036] Based on the annotation information of each sample image, the predicted image type, and the predicted device detection result, the model parameters of the preset model are updated to obtain the target model, which is used to determine the device detection result of the target device.

[0037] In one possible implementation, the model parameters of the preset model are updated based on the annotation information of each sample image, the predicted image type, and the detection results of the predictive device to obtain the target model, including:

[0038] Determine the classification loss function, the first weight of the classification loss function, the detection loss function, and the second weight of the detection loss function;

[0039] The target loss function is determined based on the classification loss function, the first weight, the detection loss function, and the second weight;

[0040] The loss value is determined based on the annotation information of each sample image, the predicted image type, the detection result of the predictive device, and the target loss function;

[0041] The model parameters of the preset model are updated based on the loss value to obtain the target model.

[0042] In one possible implementation, obtaining sample data includes:

[0043] Acquire an initial image of the sample device;

[0044] The initial image is subjected to image enhancement processing to obtain the multiple sample images; the image enhancement processing includes at least one of the following: spatial transformation processing, color perturbation processing, or hybrid enhancement processing;

[0045] Obtain the annotation information for each sample image;

[0046] The sample data is obtained based on the multiple sample images and the annotation information of each sample image.

[0047] Thirdly, embodiments of this application provide a device testing apparatus, the apparatus comprising:

[0048] The first acquisition module is used to acquire the target image corresponding to the target device;

[0049] The first processing module is used to process the multi-scale features of the target image to obtain the channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations.

[0050] The first detection module is used to determine the device detection result of the target device based on the channel attention features and the spatial attention features.

[0051] In one possible implementation, the first processing module is specifically used for:

[0052] Obtain the merged features and pooled features of the multi-scale features;

[0053] The merged features are subjected to channel attention processing to obtain channel attention features;

[0054] Spatial attention features are obtained by performing spatial attention processing on the pooling features.

[0055] In one possible implementation, the first processing module is specifically used for:

[0056] The average merged features are processed by a multilayer perceptron to obtain the first merged feature;

[0057] The second merged feature is obtained by processing the maximum merged feature using the multilayer perceptron.

[0058] The first merged feature and the second merged feature are merged to obtain the first target feature;

[0059] The channel attention features are obtained by processing the first target feature using a preset activation function.

[0060] In one possible implementation, the first processing module is specifically used for:

[0061] The average pooling feature and the max pooling feature are concatenated to obtain the second target feature;

[0062] The second target feature is convolved to obtain the third target feature;

[0063] The spatial attention features are obtained by processing the third target features using a preset activation function.

[0064] In one possible implementation, the first detection module is specifically used for:

[0065] The channel attention features and the spatial attention features are fused to obtain fused attention features;

[0066] Based on the fusion attention features, the image type of the target image is determined, and the image type includes one of the following: the front of the target device, the back of the target device, and the side of the target device;

[0067] The device detection result is determined based on the fused attention features and the image type.

[0068] In one possible implementation, the first processing module is specifically used for:

[0069] The target image is subjected to multi-scale feature extraction processing using a target model to obtain the multi-scale features;

[0070] The multi-scale features are processed by the target model to obtain the channel attention features and the spatial attention features;

[0071] Based on the channel attention features and the spatial attention features, the device detection result of the target device is determined, including:

[0072] The target model is used to process the channel attention features and the spatial attention features to obtain the device detection results.

[0073] Fourthly, embodiments of this application provide a model training apparatus, the apparatus comprising:

[0074] The second acquisition module is used to acquire sample data, which includes multiple sample images of the sample device and annotation information of each sample image. The annotation information includes the type of the labeled image and the detection result of the labeled device.

[0075] The third acquisition module is used to acquire multi-scale features of each sample image through the feature extraction module of the preset model;

[0076] The second processing module is used to process the multi-scale features of each sample image through the attention module of the preset model to obtain the fused attention features of each sample image.

[0077] The second detection module is used to process the fusion attention features of each sample image through the multi-task module of the preset model to obtain the predicted image type and the prediction device detection result of each sample image.

[0078] The update module is used to update the model parameters of the preset model based on the annotation information of each sample image, the predicted image type, and the predicted device detection result, so as to obtain the target model. The target model is used to determine the device detection result of the target device.

[0079] In one possible implementation, the update module is specifically used for:

[0080] Determine the classification loss function, the first weight of the classification loss function, the detection loss function, and the second weight of the detection loss function;

[0081] The target loss function is determined based on the classification loss function, the first weight, the detection loss function, and the second weight;

[0082] The loss value is determined based on the annotation information of each sample image, the predicted image type, the detection result of the predictive device, and the target loss function;

[0083] The model parameters of the preset model are updated based on the loss value to obtain the target model.

[0084] In one possible implementation, the second acquisition module is specifically used for:

[0085] Acquire an initial image of the sample device;

[0086] The initial image is subjected to image enhancement processing to obtain the multiple sample images; the image enhancement processing includes at least one of the following: spatial transformation processing, color perturbation processing, or hybrid enhancement processing;

[0087] Obtain the annotation information for each sample image;

[0088] The sample data is obtained based on the multiple sample images and the annotation information of each sample image.

[0089] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0090] The memory stores computer programs;

[0091] The processor is configured to perform the method described in either the first aspect or the second aspect via the computer program.

[0092] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a controller, are used to implement the method described in any one of the first and / or second aspects.

[0093] In a seventh aspect, embodiments of this application provide a computer program product, characterized in that it includes a computer program that, when executed by a controller, implements the method described in any one of the first and / or second aspects.

[0094] The device detection method and model training method provided in this application acquire a target image corresponding to a target device; process the multi-scale features of the target image to obtain channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations; based on the channel attention features and spatial attention features, the device detection result of the target device is determined. By acquiring the channel attention features of the target image, the globally important features of the target image can be comprehensively captured; by acquiring the spatial attention features of the target image, the minute details of the feature image can be captured, improving the accuracy of detecting minute devices. Attached Figure Description

[0095] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0096] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0097] Figure 2 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 1 ;

[0098] Figure 3 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 2 ;

[0099] Figure 4 This is a schematic diagram of the path aggregation network structure provided in an embodiment of this application;

[0100] Figure 5 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0101] Figure 6 This is a schematic diagram of the device detection apparatus provided in the embodiments of this application;

[0102] Figure 7 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application;

[0103] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0104] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0105] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0106] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0107] In modern manufacturing processes, the requirements for quality control of manufactured components are becoming increasingly stringent. The quality of components is crucial to the performance of products, therefore, defect detection of manufactured components is necessary.

[0108] In related technologies, device images are input into a trained defect detection model for defect detection. However, these defect detection methods use fixed-size feature maps during model training and can only detect devices with single structures. This results in low detection accuracy when detecting tiny or complex devices.

[0109] In this application embodiment, a device detection method and a model training method are proposed. By acquiring the channel attention features of the target image, the global important features of the target image can be comprehensively captured. By acquiring the spatial attention features of the target image, the minute details of the feature image can be captured, which can improve the accuracy of detecting minute devices.

[0110] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0111] It should be noted that the execution entity in each embodiment of this application can be a server, processor, microprocessor, etc. The specific execution entity in each embodiment of this application is not limited, and it can be selected and set according to actual needs. In the following embodiments, a server is used as an example of the execution entity, which does not constitute a limitation on the actual execution entity.

[0112] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1 As shown, it includes target device 11 and server 12.

[0113] The target device 11 is the device that needs to be defect-detected. The target device 11 can be various types of devices, such as chip connectors, miniature relays, probes, micro gears, etc. The target device 11 can include multiple surfaces, such as the front, back, and sides.

[0114] Server 12 is a server for performing defect detection on target device 11. Server 12 can perform defect detection on target device 11 based on images of various sides of target device 11 and output the detection results of target device 11. The detection results output by server 12 may include, for example, the presence of defects on the front of the chip connector, or the absence of defects in the miniature relay.

[0115] The solutions of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0116] Figure 2 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 1 ,like Figure 2 As shown, the method includes:

[0117] S101. Obtain the target image corresponding to the target device.

[0118] The target device is the device that needs to be defect-detected. The target device can be of various types; for example, target device 11 can be a chip connector, a miniature relay, a probe, a micro gear, etc. The target device can include multiple surfaces, such as the front, back, and sides.

[0119] A target image is an image obtained by photographing a target device, and it is used for defect detection on the target device. For example, a target image can be an image obtained by using devices such as infrared sensors, 3D scanners, and cameras. Any target device may include multiple target images, which can be images corresponding to different sides of the target device. For example, target images of a target device may include a front image, a side image, and a back image of the target device.

[0120] S102. Process the multi-scale features of the target image to obtain the channel attention features and spatial attention features of the target image.

[0121] Multi-scale features are features obtained by extracting features from a target image. Multi-scale features of a target image can be obtained through convolution operations. Different convolutional layers correspond to features of different sizes. For example, multi-scale features may include features with a resolution of 80×80, features with a resolution of 40×40, features with a resolution of 20×20, etc.

[0122] Channel attention features are used to indicate the characteristics of a target device in multiple feature channels. Channel attention features can be obtained by processing the importance weights of different channels in multi-scale features.

[0123] Spatial attention features are used to indicate features at multiple locations of a target device. Spatial attention features can be obtained by processing the importance weights of different locations in the multi-scale features.

[0124] S103. Determine the device detection results of the target device based on the channel attention characteristics and spatial attention characteristics.

[0125] Device inspection results can include whether the target device has defects, the location of the defects, and the type of defects. The location of the defects could be, for example, the coordinates of a surface on the target device; the type of defects could be, for example, cracks, device deformation, dimensional non-compliance, or surface contamination.

[0126] For example, when the target device is a chip connector, the device inspection result can be that there are no defects on the front side of the chip connector; or that there are defects on the back side of the chip connector, with the coordinates of the defect location being (X, Y), and the defect type being a crack, etc.

[0127] The device detection method and model training method provided in this application acquire a target image corresponding to a target device; process the multi-scale features of the target image to obtain channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations; based on the channel attention features and spatial attention features, the device detection result of the target device is determined. By acquiring the channel attention features of the target image, the globally important features of the target image can be comprehensively captured; by acquiring the spatial attention features of the target image, the minute details of the feature image can be captured, improving the accuracy of detecting minute devices.

[0128] The process of determining the device detection results of the target device through multi-scale features is described in detail below.

[0129] Figure 3 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 2 ,like Figure 3 As shown, the method includes:

[0130] S201. Obtain merged and pooled features from multi-scale features.

[0131] Merged features are features obtained by aggregating the spatial information of input features through average pooling (AvgPool) and max pooling (MaxPool), respectively. Merged features can include average merged features and max merged features, and the input features can be any feature from the multi-scale features. For any scale feature, performing an average pooling operation on that scale feature yields the average merged feature, and performing a max pooling operation on that scale feature yields the max merged feature.

[0132] The average pooling feature is used to represent the global average activation intensity of a channel. For example, the size of a feature at a certain scale is (C×H×W), where C is the number of channels in the feature, H is the height of the feature, and W is the width of the feature. By performing an average pooling operation on this feature and averaging the values ​​of all spatial locations for each channel, the average pooling feature is obtained, and the size of the average pooling feature is (C×1×1).

[0133] The maximum pooling feature is used to capture the most salient feature response. For example, if a feature of size (C×H×W) is performed at a certain scale, the maximum pooling operation is performed on this feature, taking the maximum value at all spatial locations of each channel, to obtain the maximum pooling feature, which has a size of (C×1×1).

[0134] Pooling features are obtained by aggregating input features along the channel dimension using average pooling and max pooling. Pooling features can include average pooling features and max pooling features. For any scale feature, performing average pooling on that scale feature yields average pooling features, and performing max pooling on that scale feature yields max pooling features.

[0135] Average pooling features are used to reflect the global spatial response. For example, the average pooling feature is obtained by averaging all channel values ​​at each spatial location (h, w), and the size of the average pooling feature is (1×H×W).

[0136] Max pooling is used to capture salient local features. For example, if a feature at a certain scale has a size of (C×H×W), max pooling is performed on this feature, taking the maximum value of all channel values ​​at each spatial location (h, w) to obtain the max pooled feature, which has a size of (1×H×W).

[0137] S202. Perform channel attention processing on the merged features to obtain channel attention features.

[0138] Channel attention features can be obtained as follows: the average merged feature is processed by a multi-layer perceptron (MLP) to obtain the first merged feature; the maximum merged feature is processed by a multi-layer perceptron to obtain the second merged feature; the first merged feature and the second merged feature are merged to obtain the first target feature; the first target feature is processed by a preset activation function to obtain the channel attention features.

[0139] The first and second merged features can be obtained as follows: The MLP can generate channel weights through a fully connected layer, compressing the number of channels in the average merged feature and the maximum merged feature from C to C / r, thus obtaining the second merged feature and the maximum merged feature. Here, r is the dimensionality reduction coefficient of the MLP, used to reduce parameter overhead. For example, the dimensions of the average merged feature and the maximum merged feature before inputting into the MLP are (C×H×W), and the dimensions after dimensionality reduction by the MLP are (C / r×H×W).

[0140] After obtaining the first merged feature and the second merged feature, the first target feature can be obtained by summing the elements of the first merged feature and the second merged feature. Then, the first target feature is normalized to the range [0,1] by the activation function to obtain the channel attention feature.

[0141] The process of obtaining channel attention features can be represented by formula (1):

[0142]

[0143] in, The features are used as input to the multilayer perceptron. For channel attention features, This indicates the average pooling operation. This indicates a max pooling operation. This indicates a fully connected operation. For average merging characteristics, For the largest merge feature, For activation function, and To pass The generated channel weights.

[0144] S203. Perform spatial attention processing on the pooling features to obtain spatial attention features.

[0145] Spatial attention features can be obtained as follows: concatenate the average pooling features and the max pooling features to obtain the second target feature; perform convolution on the second target feature to obtain the third target feature; process the third target feature through a preset activation function to obtain the spatial attention features.

[0146] The process of acquiring spatial attention features can be represented by formula (2):

[0147]

[0148] in, As input features, Spatial attention features This indicates the average pooling operation. This indicates a max pooling operation. Indicates use Convolution operations are performed using convolution kernels of varying sizes. For average pooling characteristics, For max pooling features, This is the activation function.

[0149] S204. The channel attention features and spatial attention features are fused to obtain the fused attention features.

[0150] The fused attention features can be obtained by designing two learnable attention matrices. and The results of multiplying the attention features by the channel attention features and spatial attention features respectively are then summed to obtain the fused attention features. Among these, the attention matrix... and The importance of adjusting channel attention and spatial attention. and It can be learned by convolutional layers and dynamically adjusted during model training using a multi-task loss function. The process of determining the fused attention features can be represented by formula (3):

[0151]

[0152] in, To integrate attention features, For input features.

[0153] S205. Determine the image type of the target image based on the fusion attention features.

[0154] The image type includes one of the following: the front of the target device, the back of the target device, and the side of the target device.

[0155] The image type of the target image can be determined by processing the fused attention features through a decoder. The decoder can include five layers. The shallow decoder (layers 1, 2, and 3) performs feature reconstruction and spatial information recovery on the fused attention features to obtain the first feature. The size of the first feature is 1 / 8 of the target image size. The image type of the target image is determined based on the first feature.

[0156] The image type of a target image can be determined as follows: The deep decoder (layers 4 and 5) upsamples the first feature twice to obtain a second feature half the size of the target image. This second feature is then passed through an average pooling layer and a fully connected layer to output the image type. The average pooling layer compresses the spatial dimensions (height and width) of the second feature into a single scalar. The fully connected layer non-linearly combines the output of the average pooling layer using a weight matrix to generate an output vector matching the number of categories. The output vector is then converted into a probability distribution using an activation function, from which the image type is obtained. The number of output units equals the number of target image types. For example, if the target image of a device includes three types: front, back, and side, then the number of output units is also 3.

[0157] S206. Determine the device detection results based on the fused attention features and image type.

[0158] The first feature can be detected using a path aggregation network to obtain the device detection result. Specifically, the path aggregation network uses a bottom-up feature pyramid network combined with a top-down feature pyramid to detect the first feature.

[0159] The following section, with reference to the accompanying diagram, illustrates a specific example of how to obtain device detection results through a path aggregation network.

[0160] Figure 4This is a schematic diagram of the path aggregation network structure provided in the embodiments of this application, such as... Figure 4 As shown, it includes a top-down feature pyramid and a bottom-up feature pyramid. The top-down feature pyramid transmits semantic information from the fifth layer to the third layer and then transmits the semantic information to the bottom-up feature pyramid. The bottom-up feature pyramid performs feature detection from the third layer to the fifth layer.

[0161] In the top-down feature pyramid, the features from the fifth layer to the third layer are represented as C5, C4, and C3, respectively. The size of feature C3 is 1 / 8 of the sample image, feature C4 is 1 / 16, and feature C5 is 1 / 32. Starting with C5, C5 is upsampled by a factor of 2, increasing its size to match C4, and then concatenated with C4 (i.e., adjusting the number of channels through a 1×1 convolution) to obtain P4. P4 is then upsampled by a factor of 2, increasing its size to match C3, and then horizontally concatenated with C3 to obtain P3.

[0162] In one possible implementation, the top-down feature pyramid passes the features of each layer to the corresponding layers in the bottom-up feature pyramid. For example, the top-down feature pyramid passes C5 to the fifth layer in the bottom-up feature pyramid, the top-down feature pyramid passes P4 to the fourth layer in the bottom-up feature pyramid, and the top-down feature pyramid passes P3 to the third layer in the bottom-up feature pyramid.

[0163] In one possible implementation, the top-down feature pyramid simultaneously transmits features from each layer to the bottom-up feature pyramid. For example, the top-up feature pyramid simultaneously transmits C5, P4, and P3 to the bottom-up feature pyramid.

[0164] In the bottom-up feature pyramid, P3, transmitted from the top-down pyramid, is first downsampled to reduce its resolution so that its size is the same as P4, and then concatenated with P4 to obtain N4; then N4 is downsampled to reduce its resolution so that its size is concatenated with C5 to obtain N5.

[0165] The detection head performs detection based on the output of each layer in the bottom-up feature pyramid, obtaining device detection results corresponding to different sized features. For example, detecting feature P3 yields the detection result corresponding to feature P3, detecting feature N4 yields the detection result corresponding to feature N4, and detecting feature N5 yields the detection result corresponding to feature N5.

[0166] In this embodiment of the application, by distinguishing each surface of the device, the specific structure corresponding to each surface of the device can be clearly identified, enabling detection to be performed on specific structures, thereby improving the detection accuracy. By constructing bidirectional feature fusion, low-level details and high-level semantics are fully combined, significantly improving the detection accuracy of small targets and complex backgrounds.

[0167] Optionally, the device detection method provided in this application embodiment further includes training a preset model to obtain a target model, and obtaining the device detection result through the target model.

[0168] The model training method provided in the embodiments of this application will be described in detail below.

[0169] Figure 5 This is a flowchart illustrating the model training method provided in the embodiments of this application, as shown below. Figure 5 As shown, it includes:

[0170] S301. Obtain sample data.

[0171] The sample data includes multiple sample images of the sample device and annotation information for each sample image. The annotation information includes the type of the labeled image and the detection result of the labeled device.

[0172] Sample images are used to train the model. Multiple initial images can be obtained by taking pictures of each device using a mobile or multi-angle camera system. Image enhancement processing is then performed on these initial images to obtain sample images.

[0173] Initial images can be obtained as follows: Divide each device into regions, and for any given region, magnify it N times from different angles to capture multiple initial images, where N is a positive integer greater than or equal to 1, for example, N can be 30. The initial images obtained through this method can more accurately depict the details of the device, ensuring effective detection of small targets.

[0174] In one possible implementation, to avoid the impact of changes in lighting on image quality and thus ensure the accuracy of defect detection, devices such as infrared sensors, 3D scanners, and RGB cameras can be used to acquire device images separately in environments with unstable lighting. The device images are then fused together to obtain a sample image.

[0175] The sample images are obtained by enhancing the initial image. Image processing can include spatial transformation, color perturbation, or hybrid enhancement. Spatial transformation, such as rotating, scaling, and translating the initial image, introduces slight deformations to simulate changes during capture. Color perturbation, such as adjusting brightness, contrast, and saturation, and adding random noise, simulates the effects of lighting or sensor noise on the initial image during capture. Hybrid enhancement involves blending the initial image with images obtained through spatial transformation and color perturbation. Image enhancement of the initial image can increase the number of sample images containing device defects, thus improving the model's generalization ability.

[0176] The annotation information for sample images can include the image type and the detection results of the annotated devices. The image type can include various sides of different types of devices, such as chip connectors, miniature relays, probes, and micro gears. Image types include, for example, the front side of a chip connector, the back side of a chip connector, and the front side of a miniature relay.

[0177] Image types can be labeled, such as the front image of connector 1, the back image of connector 2, etc.; device inspection results can be labeled, such as no defects on the front of connector 1, defects on the back of connector 1, defects in the upper left corner of the front of connector 1, cracks in connector 1, etc.

[0178] S302. Obtain multi-scale features of each sample image through the feature extraction module of the preset model.

[0179] The preset model is a model that requires sample images and their annotation information for training. The preset model can be used to classify sample images and detect defects in sample images through model training.

[0180] The pre-defined model may include a feature extraction module, which is used to obtain multi-scale features of each sample image. Multi-scale features of the sample images can be obtained through convolution operations, with different convolutional layers corresponding to features of different sizes. Examples of multi-scale features include features with a resolution of 80×80, 40×40, and 20×20, etc.

[0181] S303. The multi-scale features of each sample image are processed by the attention module of the preset model to obtain the fused attention features of each sample image.

[0182] The preset model may include an attention module, which is used to process the multi-scale features of the sample image to obtain the fused attention features of the sample image.

[0183] Attention modules can include channel attention modules and spatial attention modules. Channel attention modules can learn the importance weights of different feature channels at multiple scales to obtain channel attention features; spatial attention modules can learn the importance weights of spatial locations of features at multiple scales to obtain spatial attention features.

[0184] Fusion attention features are features obtained by fusing channel attention features and spatial attention features. By combining channel and spatial information, fusion attention features enable the pre-defined model to capture important features more comprehensively. For example, the channel attention module can focus on specific colors and textures, while the spatial attention module can focus on defect locations, thereby improving detection accuracy.

[0185] Among them, the method for obtaining fused attention features and Figure 3 The method described above is the same as that described above, so it will not be repeated here.

[0186] S304. The multi-task module of the preset model processes the fusion attention features of each sample image to obtain the predicted image type and the prediction device detection result of each sample image.

[0187] The pre-defined model may include a multi-task module, which processes the fused attention features of each sample image to obtain the prediction type and detection result of each sample image. The multi-task module includes a classification module and a detection module.

[0188] The classification module processes the fused attention features to obtain the predicted image type of the sample images. The predicted image type includes the various faces of different types of devices; for example, sample image 1 is predicted as the front of connector 1, and sample image 2 is predicted as the back of connector 2. Different types of devices have different structures, and even within the same type of device, the structures of each face differ. By distinguishing the types of sample images, the device structure corresponding to each predicted image type can be clearly identified. Specific training can then be performed on this structure to improve the model's detection performance.

[0189] The detection module processes the predicted image type and fused attention features to obtain the predicted device detection results. These results can include whether the device has defects, the location of the defects, and the type of defects. For example, the presence of defects could be indicated by the absence of defects on the front of connector 1, while connector 2 has defects on the back. The location of defects could be indicated by the location of a defect on the back of connector 2 at coordinates (X, Y). Defect types could include cracks, deformation, non-compliant dimensions, and surface contamination.

[0190] Among them, the methods for obtaining the predicted image type and the prediction device detection results are similar to... Figure 3 and Figure 4 The method described above is the same as that described above, so it will not be repeated here.

[0191] S305. Based on the annotation information of each sample image, the predicted image type, and the detection results of the predictive device, update the model parameters of the preset model to obtain the target model.

[0192] The target model is used to determine the device detection results of the target device.

[0193] The model parameters of the preset model are those that can be adjusted during the training process. By adjusting the model parameters, the model can be guided to gradually approach its optimal performance state during training, thereby obtaining the target model.

[0194] Model parameters can be adjusted by determining a loss function, which measures the difference between the predicted and actual results, reflecting the accuracy and error of the pre-set model's predictions. During the training process of the pre-set model, continuously adjusting the model parameters to minimize the loss function allows the model to converge quickly, improving its overall performance.

[0195] The target model can be obtained as follows: determine the classification loss function, the first weight of the classification loss function, the detection loss function, and the second weight of the detection loss function; determine the target loss function based on the classification loss function, the first weight, the detection loss function, and the second weight; determine the loss value based on the annotation information of each sample image, the predicted image type, the predicted device detection result, and the target loss function; update the model parameters of the preset model based on the loss value to obtain the target model.

[0196] The classification loss function is used for supervised training of classification tasks. It can be determined by formula (4):

[0197]

[0198] in, The number of device label image types, including multiple categories of devices and multiple faces of each device; The raw input during training; For image types, such as the front image of device 1, the back image of device 2, etc.; This indicates that the sigmoid function is applied to the model input. The results above.

[0199] The detection loss function is used for supervised training of the detection task. It can be determined by formula (5):

[0200]

[0201] in, For regression loss function, These are the weights of the regression loss function; For confidence loss function, These are the weights of the confidence loss function; For defect type loss function, The weights of the defect type loss function are used to balance the importance of each loss function. For example, the weight of the regression loss function can be 0.05, the weight of the confidence loss function can be 1, and the weight of the defect type loss function can be 0.5, etc.

[0202] Regression loss function It can be determined by formula (6):

[0203]

[0204] in, This indicates the number of grids that the first feature is divided into. This indicates the number of predicted boxes on each grid cell. This indicates whether the prediction box contains the defect target; 1 indicates it contains the defect, and 0 indicates it does not. This is an improved bounding box regression loss function used to differentiate between the predicted and ground truth bounding boxes.

[0205] Confidence loss function and defect type loss function The numerically stable binary cross-entropy (BCEWithLogistsLOSS) calculation method is adopted, and the calculation formulas are shown in formula (7) and formula (8) respectively. The confidence loss function is used to measure whether there is a defective target in the bounding box, and the defect type loss function is used to measure the specific category of the defect in the bounding box.

[0206]

[0207] in, and This indicates that the predicted bounding box contains a defect target, while and This indicates that the predicted bounding box does not contain defect targets. This represents the confidence level of the true predicted bounding box. This indicates the confidence level of the predicted bounding box.

[0208]

[0209] in, Indicates the defect category, Indicates the true category of the defect within the predicted bounding box. Indicates the prediction category.

[0210] After determining the classification loss function and the detection loss function, a first weight is designed for the classification loss function and a second weight is designed for the detection loss function. Based on the first and second weights, the classification loss function and the detection loss function are weighted and summed to obtain the target loss function. The target loss function can be determined by formula (9):

[0211]

[0212] in, These are the weights of the classification loss function, i.e., the first weights; The weights for the detection loss function are called the second weights. and These are hyperparameters used to adjust the weighting of the detection loss function and the classification loss function, respectively. and The value of can be adjusted during training using cross-validation to optimize the performance of the pre-defined model. For example, The initial value can be set to 0.3. The initial value can be set to 0.7, and during training, it can be increased by... value, decrease The value, or decrease value, increase The values ​​are adjusted to the preset model, and the performance of the preset model under each condition is determined. The weight value with the best performance is taken as the final weight value to obtain the trained model.

[0213] Furthermore, after obtaining the trained model, it can be fine-tuned and updated based on the detection results of the predicted device to obtain the target model.

[0214] When using a trained model for model detection, manual verification can be used to determine if there are any anomalies such as missed detections or false detections. If anomalies are found, the abnormal data is transferred to the model database via FTP protocol. The model automatically labels the abnormal data, and then the labeling results are manually verified to obtain new sample images. Based on the new sample images, the trained model is fine-tuned or retrained. Finally, based on the model recall and accuracy metrics, the fine-tuned model or the retrained model is determined as the target model.

[0215] Fine-tuning the trained model involves, for example, adding new sample images to the original sample images, calculating a new loss function, and adjusting the parameters in the model. Retraining the model can also be done, for example, by retraining a detection model using new sample images.

[0216] Target Model It can be determined by formula (10):

[0217]

[0218] in, This represents the fine-tuned model. This indicates a retrained model. The recall rate of the fine-tuned model. The recall rate of the retrained model. The accuracy of the fine-tuned model. The accuracy of the retrained model.

[0219] In formula (10), recall is weighted at 0.55 and precision at 0.45, indicating that both the cost of missed detections and detection accuracy are considered, with the cost of missed detections being higher than the accuracy. This is achieved by... Once the loss value is determined, the target model is determined. When the loss value of the fine-tuned model is the same as the loss value of the retrained model, the model with higher recall is selected as the target model. If the recall is the same, the model with higher accuracy is selected as the target model.

[0220] The model training method provided in this application uses a preset model to process multi-scale features of sample images to obtain fused attention features of the sample images, which can simultaneously focus on global features and minute details, improving the accuracy of detecting minute devices. By designing a multi-task module to process the fused attention features of the sample images, the method obtains the sample image type and predicted device detection results, enabling simultaneous classification detection and defect detection of sample images, thus improving the model's feature extraction efficiency and detection accuracy. By adjusting the model parameters during training, the overall performance of the target model can be further improved.

[0221] Optionally, during the process of using the target model to detect devices, abnormal situations such as missed detections and false detections can be continuously collected, and abnormal data can be added to the sample images. The target model can be fine-tuned or incrementally trained periodically to continuously optimize the detection performance of the target model.

[0222] Furthermore, the device detection method and model training method provided in this application embodiment can be applied to precision devices involved in the communications industry, as well as to other devices with similar structures, and have strong adaptability.

[0223] Furthermore, the device detection method and model training method provided in this application embodiment can be deployed in embedded edge devices to reduce network transmission latency and improve real-time performance and detection efficiency.

[0224] Figure 6This is a schematic diagram of the device testing apparatus provided in the embodiments of this application, as shown below. Figure 6 As shown, the device includes:

[0225] The first acquisition module 61 is used to acquire the target image corresponding to the target device;

[0226] The first processing module 62 is used to process the multi-scale features of the target image to obtain the channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations.

[0227] The first detection module 63 is used to determine the device detection result of the target device based on the channel attention features and the spatial attention features.

[0228] In one possible implementation, the first processing module 62 is specifically used for:

[0229] Obtain the merged features and pooled features of the multi-scale features;

[0230] The merged features are subjected to channel attention processing to obtain channel attention features;

[0231] Spatial attention features are obtained by performing spatial attention processing on the pooling features.

[0232] In one possible implementation, the first processing module 62 is specifically used for:

[0233] The average merged features are processed by a multilayer perceptron to obtain the first merged feature;

[0234] The second merged feature is obtained by processing the maximum merged feature using the multilayer perceptron.

[0235] The first merged feature and the second merged feature are merged to obtain the first target feature;

[0236] The channel attention features are obtained by processing the first target feature using a preset activation function.

[0237] In one possible implementation, the first processing module 62 is specifically used for:

[0238] The average pooling feature and the max pooling feature are concatenated to obtain the second target feature;

[0239] The second target feature is convolved to obtain the third target feature;

[0240] The spatial attention features are obtained by processing the third target features using a preset activation function.

[0241] In one possible implementation, the first detection module 63 is specifically used for:

[0242] The channel attention features and the spatial attention features are fused to obtain fused attention features;

[0243] Based on the fusion attention features, the image type of the target image is determined, and the image type includes one of the following: the front of the target device, the back of the target device, and the side of the target device;

[0244] The device detection result is determined based on the fused attention features and the image type.

[0245] In one possible implementation, the first processing module 62 is specifically used for:

[0246] The target image is subjected to multi-scale feature extraction processing using a target model to obtain the multi-scale features;

[0247] The multi-scale features are processed by the target model to obtain the channel attention features and the spatial attention features;

[0248] Based on the channel attention features and the spatial attention features, the device detection result of the target device is determined, including:

[0249] The target model is used to process the channel attention features and the spatial attention features to obtain the device detection results.

[0250] Figure 7 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application, as shown below. Figure 7 As shown, the device includes:

[0251] The second acquisition module 71 is used to acquire sample data, which includes multiple sample images of the sample device and annotation information of each sample image. The annotation information includes the type of the labeled image and the detection result of the labeled device.

[0252] The third acquisition module 72 is used to acquire multi-scale features of each sample image through the feature extraction module of the preset model;

[0253] The second processing module 73 is used to process the multi-scale features of each sample image through the attention module of the preset model to obtain the fused attention features of each sample image.

[0254] The second detection module 74 is used to process the fusion attention features of each sample image through the multi-task module of the preset model to obtain the predicted image type and the prediction device detection result of each sample image.

[0255] The update module 75 is used to update the model parameters of the preset model based on the annotation information of each sample image, the predicted image type and the predicted device detection result, to obtain the target model, which is used to determine the device detection result of the target device.

[0256] In one possible implementation, update module 75 is specifically used for:

[0257] Determine the classification loss function, the first weight of the classification loss function, the detection loss function, and the second weight of the detection loss function;

[0258] The target loss function is determined based on the classification loss function, the first weight, the detection loss function, and the second weight;

[0259] The loss value is determined based on the annotation information of each sample image, the predicted image type, the detection result of the predictive device, and the target loss function;

[0260] The model parameters of the preset model are updated based on the loss value to obtain the target model.

[0261] In one possible implementation, the second acquisition module 71 is specifically used for:

[0262] Acquire an initial image of the sample device;

[0263] The initial image is subjected to image enhancement processing to obtain the multiple sample images; the image enhancement processing includes at least one of the following: spatial transformation processing, color perturbation processing, or hybrid enhancement processing;

[0264] Obtain the annotation information for each sample image;

[0265] The sample data is obtained based on the multiple sample images and the annotation information of each sample image.

[0266] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 8 As shown, the electronic device 800 includes a memory 801 and a processor 802.

[0267] The memory 801 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0268] The memory 801 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage device.

[0269] The processor 802 is used to execute computer execution instructions stored in the memory 801 to implement the model training method and / or device detection method described in the foregoing method embodiments. The processor 501 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0270] Optionally, the electronic device 800 may also include a communication interface 803. In specific implementations, if the communication interface 803, memory 801, and processor 802 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0271] Optionally, in a specific implementation, if the communication interface 803, memory 801, and processor 802 are integrated on a single chip, then the communication interface 803, memory 801, and processor 802 can communicate through an internal interface.

[0272] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory, a disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0273] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of an electronic device can read the executable instructions from the readable storage medium, and the processor executes the executable instructions to cause the electronic device to implement the model training and / or device detection methods provided in the various embodiments described above.

[0274] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0275] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0276] In this application, "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects have an "or" relationship.

[0277] "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b, and c, where each of a, b, and c can be an element itself or a set containing one or more elements.

[0278] In this application, "at least one" means one or more. "More than one" means two or more. The descriptions of "first," "second," etc., appearing in the embodiments of this application are only for illustration and to distinguish the described objects, and have no order, nor do they indicate a special limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application. For example, "first threshold" and "second threshold" are only used to distinguish different thresholds, and do not indicate that the size, priority, or importance of these two thresholds are different.

[0279] In this application, terms such as "exemplary," "in some embodiments," and "in other embodiments" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the term "exemplary" is used to present the concept in a specific manner.

[0280] In this application, the terms "of," "corresponding," and "related" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their meanings are consistent. Similarly, in the embodiments of this application, "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their meanings are consistent. For example, transmission can include sending and / or receiving, and can be a noun or a verb.

[0281] In this application, "equal to" can be used with "less than" or "greater than", but not simultaneously with both. When "equal to" is used with "less than", it applies to the technical solution adopted by "less than". When "equal to" is used with "greater than", it applies to the technical solution adopted by "greater than".

[0282] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A device testing method, characterized in that, include: Obtain the target image corresponding to the target device; The multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image. The channel attention features are used to indicate the features of the target device in multiple feature channels, and the spatial attention features are used to indicate the features of the target device at multiple locations. The device detection result of the target device is determined based on the channel attention features and the spatial attention features.

2. The method according to claim 1, characterized in that, The multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image, including: Obtain the merged features and pooled features of the multi-scale features; The merged features are subjected to channel attention processing to obtain channel attention features; Spatial attention features are obtained by performing spatial attention processing on the pooling features.

3. The method according to claim 2, characterized in that, The merging characteristics include the average merging characteristic and the maximum merging characteristic; The merged features are subjected to channel attention processing to obtain channel attention features, including: The average merged features are processed by a multilayer perceptron to obtain the first merged feature; The second merged feature is obtained by processing the maximum merged feature using the multilayer perceptron. The first merged feature and the second merged feature are merged to obtain the first target feature; The channel attention features are obtained by processing the first target feature using a preset activation function.

4. The method according to claim 2 or 3, characterized in that, The pooling characteristics include average pooling characteristics and maximum pooling characteristics; Spatial attention processing is applied to the pooling features to obtain spatial attention features, including: The average pooling feature and the max pooling feature are concatenated to obtain the second target feature; The second target feature is convolved to obtain the third target feature; The spatial attention features are obtained by processing the third target features using a preset activation function.

5. The method according to any one of claims 1-4, characterized in that, Based on the channel attention features and the spatial attention features, the device detection result of the target device is determined, including: The channel attention features and the spatial attention features are fused to obtain fused attention features; Based on the fusion attention features, the image type of the target image is determined, and the image type includes one of the following: the front of the target device, the back of the target device, and the side of the target device; The device detection result is determined based on the fused attention features and the image type.

6. The method according to any one of claims 1-5, characterized in that, The multi-scale features of the target image are processed to obtain the channel attention features and spatial attention features of the target image, including: The target image is subjected to multi-scale feature extraction processing using a target model to obtain the multi-scale features; The multi-scale features are processed by the target model to obtain the channel attention features and the spatial attention features; Based on the channel attention features and the spatial attention features, the device detection result of the target device is determined, including: The target model is used to process the channel attention features and the spatial attention features to obtain the device detection results.

7. A model training method, characterized in that, include: Acquire sample data, which includes multiple sample images of the sample device and annotation information for each sample image. The annotation information includes the type of the annotated image and the detection result of the annotated device. The feature extraction module of the preset model is used to obtain multi-scale features of each sample image; The multi-scale features of each sample image are processed by the attention module of the preset model to obtain the fused attention features of each sample image. The multi-task module of the preset model processes the fusion attention features of each sample image to obtain the predicted image type and the prediction device detection result of each sample image. Based on the annotation information of each sample image, the predicted image type, and the predicted device detection result, the model parameters of the preset model are updated to obtain the target model, which is used to determine the device detection result of the target device.

8. The method according to claim 7, characterized in that, Based on the annotation information of each sample image, the predicted image type, and the detection results of the predictive device, the model parameters of the preset model are updated to obtain the target model, including: Determine the classification loss function, the first weight of the classification loss function, the detection loss function, and the second weight of the detection loss function; The target loss function is determined based on the classification loss function, the first weight, the detection loss function, and the second weight; The loss value is determined based on the annotation information of each sample image, the predicted image type, the detection result of the predictive device, and the target loss function; The model parameters of the preset model are updated based on the loss value to obtain the target model.

9. The method according to claim 7 or 8, characterized in that, Obtain sample data, including: Acquire an initial image of the sample device; The initial image is subjected to image enhancement processing to obtain the multiple sample images; the image enhancement processing includes at least one of the following: spatial transformation processing, color perturbation processing, or hybrid enhancement processing; Obtain the annotation information for each sample image; The sample data is obtained based on the multiple sample images and the annotation information of each sample image.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor; The memory stores computer programs; The processor is configured to execute the method of any one of claims 1-6 or 7-9 via the computer program.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a controller, are used to implement the method as described in any one of claims 1-6 or 7-9.

Images