Component deformation detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the application disclose a component deformation detection method and device, electronic equipment and a storage medium; including: performing feature extraction processing on a container image to obtain a first number of embedded feature sequences; performing a preset processing for a set number of times in succession on the first number of embedded feature sequences to obtain a second number of processing result features, the second number being less than the first number; performing attention mechanism processing on each processing result feature to obtain an attention result; performing full connection processing on each attention result to obtain a full connection processing result; performing up-sampling processing on the second number of full connection processing results to obtain a component recognition result image, the component recognition result image including a to-be-detected component; and performing deformation detection on the to-be-detected component to obtain a deformation detection result of the to-be-detected component. Since the shape of the to-be-detected component is obtained first, and then whether the to-be-detected component is deformed is detected, the accuracy of detection can be improved.

Description

Technical Field

[0001] This application relates to the field of computers, and specifically to a method, apparatus, electronic device, and storage medium for detecting component deformation. Background Technology

[0002] Containers, as essential loading tools in the cargo circulation process, are widely used in various transportation scenarios such as ports and railways. Container quality management is a crucial aspect of ensuring their long-term usability. The loss or deformation of container components will inevitably affect their normal use to some extent. A bent door lock rod is a relatively common deformation that impacts the normal operation of a container. A bent door lock rod may affect the closing or opening of the container doors, thus affecting cargo loading.

[0003] In existing technologies, when performing bending detection on container door lock bars, deformation detection is typically performed on photographs containing the lock bars. However, due to the environment in which the container is located, the brightness variations in the photographs are large, and the container background is complex, making deformation detection of container door lock bars difficult and resulting in low accuracy. Summary of the Invention

[0004] This application provides a component deformation detection method, apparatus, electronic device, and storage medium, which can improve the problem of low accuracy in deformation detection in the prior art.

[0005] This application provides a component deformation detection method, which includes: performing feature extraction processing on a container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located in the container; performing a preset number of consecutive processing steps on the first number of embedded feature sequences to obtain a second number of processing result features, wherein the second number is less than the first number; performing attention mechanism processing on each of the processing result features to obtain an attention result; performing fully connected processing on each of the attention results to obtain a fully connected processing result; performing upsampling processing on the second number of fully connected processing results to obtain a component recognition result image, wherein the component recognition result image includes the component to be detected; and performing deformation detection on the component to be detected to obtain a deformation detection result of the component to be detected.

[0006] This application embodiment also provides a component deformation detection device, the device comprising:

[0007] A feature extraction unit is used to perform feature extraction processing on a container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located in the container;

[0008] A preset processing unit is used to continuously perform a preset processing on the first number of embedded feature sequences a set number of times to obtain a second number of processing result features, wherein the second number is less than the first number;

[0009] An attention mechanism unit is used to perform attention mechanism processing on each of the processing result features to obtain an attention result;

[0010] A fully connected processing unit is used to perform fully connected processing on each attention result to obtain a fully connected processing result;

[0011] An upsampling processing unit is used to upsample the second number of fully connected processing results to obtain a component identification result image, wherein the component identification result image includes the component to be detected;

[0012] The deformation detection unit is used to perform deformation detection on the component to be tested and obtain the deformation detection result of the component to be tested.

[0013] In one embodiment, the feature extraction unit includes:

[0014] A segmentation processing subunit is used to segment the container image to obtain a first number of segmentation results;

[0015] The linear transformation subunit is used to perform linear transformation processing on each of the segmentation results to obtain the corresponding embedded feature sequence.

[0016] In one embodiment, the preset processing unit includes:

[0017] The intermediate result subunit is used to perform the preset processing on the first number of embedded feature sequences to obtain a third number of intermediate result features, wherein the third number is less than the first number and the third number is greater than the second number;

[0018] The processing result subunit is used to perform the preset processing on the third number of intermediate result features to obtain the second number of processing result features.

[0019] In one implementation, the intermediate result subunit includes:

[0020] The attention sub-unit is used to process each of the embedded feature sequences using an attention mechanism to obtain the attention sub-result;

[0021] A fully connected sub-unit is used to perform fully connected processing on each of the attention sub-results to obtain a fully connected processing sub-result;

[0022] The downsampling sub-unit is used to downsample the first number of fully connected processing sub-results to obtain the third number of intermediate result features.

[0023] In one embodiment, the deformation detection unit includes:

[0024] An extraction processing subunit is used to perform skeleton extraction processing on the component to be detected to obtain an extraction processing result, wherein the extraction processing result includes multiple pixels used to characterize the skeleton shape of the component to be detected;

[0025] The clustering subunit is used to perform clustering processing on the extraction results to obtain clustering results;

[0026] The baseline subunit is used to generate a deformation reference baseline for the component to be detected based on the clustering results.

[0027] The deformation detection subunit is used to determine the deformation detection result of the component to be detected based on the relative positional relationship between the plurality of pixels and the deformation reference baseline.

[0028] In one embodiment, the device further includes:

[0029] The model acquisition unit is used to acquire the component recognition model;

[0030] The model training unit is used to train the component recognition model to obtain the trained component recognition model; wherein, the steps of performing feature extraction processing on the container image to obtain a first number of embedded feature sequences, and the steps of upsampling processing on a second number of fully connected processing results to obtain a component recognition result image are executed by the trained component recognition model.

[0031] In one implementation, the model training unit includes:

[0032] The feature extraction training subunit is used to perform feature extraction processing on the training detection image to obtain a first number of training embedding feature sequences, wherein the training detection image includes a training detection component;

[0033] The training processing result subunit is used to continuously perform a preset number of processing operations on the first number of training embedded feature sequences to obtain a second number of training processing result features, wherein the second number is less than the first number.

[0034] A training attention subunit is used to process each of the training result features using an attention mechanism to obtain the training attention result.

[0035] A fully connected processing subunit is used to perform fully connected processing on each of the training attention results to obtain the training fully connected processing result;

[0036] An upsampling processing subunit is used to upsample the second number of training fully connected processing results to obtain a training component recognition result image, which includes the shape of the training detection component.

[0037] The loss function construction subunit is used to construct a loss function based on the shape of the labeled detection component, the shape of the training detection component, the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample.

[0038] The training decision subunit is used to determine that the component recognition model has been trained when the loss function meets the preset requirements, and to obtain the trained component recognition model.

[0039] In one implementation, training the decision subunit includes:

[0040] The first loss subunit is used to construct a first loss function based on the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the label category corresponding to each sample.

[0041] The second loss subunit is used to construct a second loss function based on the shape of the labeled detection component and the shape of the trained detection component;

[0042] The loss function construction sub-unit is used to construct the loss function based on the first loss function and the second loss function.

[0043] In the component deformation detection method provided in this application embodiment, feature extraction can be performed on a container image to obtain a first number of embedded feature sequences. Then, the first number of embedded feature sequences are subjected to a predetermined number of consecutive preset processing steps to obtain a second number of processed result features. Each processed result feature is sequentially processed using an attention mechanism and then fully connected processing, resulting in a second number of fully connected processing results. Subsequently, the second number of fully connected processing results are upsampled to obtain a component recognition result image including the component to be detected. Then, deformation detection is performed on the component to be detected included in the component recognition result image to obtain the deformation detection result of the component to be detected.

[0044] In this embodiment, for a container image, the components included in the container image can be identified sequentially through feature extraction processing, a set number of preset processing steps, attention mechanism processing, fully connected processing, and upsampling processing. The identification result includes the component to be detected. Since the component to be detected is identified first, and the accurate shape of the component to be detected is obtained, the deformation of the component to be detected is then detected. Therefore, this embodiment can improve the accuracy of deformation detection of the component to be detected. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1a This is a schematic diagram illustrating a usage scenario of the component deformation detection method provided in this application embodiment;

[0047] Figure 1b This is a schematic flowchart of a component deformation detection method provided in an embodiment of this application;

[0048] Figure 1c A schematic diagram illustrating a specific implementation of segmentation processing for container images is shown;

[0049] Figure 1d Image (1) shows a container corresponding to a specific embodiment;

[0050] Figure 1d (2) shows Figure 1d The component recognition result image corresponding to the container image in (1);

[0051] Figure 1d (3) shows a schematic diagram of deformation detection of the component to be tested;

[0052] Figure 1e Image (1) shows a container corresponding to another specific embodiment;

[0053] Figure 1e (2) shows Figure 1e The component recognition result image corresponding to the container image in (1);

[0054] Figure 1e (3) shows Figure 1e The extraction and processing results corresponding to the component identification result image in (2) of the image;

[0055] Figure 2This is a schematic flowchart of a component deformation detection method provided in another embodiment of this application;

[0056] Figure 3 This is a schematic diagram of a component deformation detection device provided in one embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] This application provides a component deformation detection method, apparatus, electronic device, and storage medium.

[0060] Specifically, the deformation detection device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC). The server can be a single server or a server cluster consisting of multiple servers.

[0061] In some embodiments, the component deformation detection device can also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the component deformation detection method of this application.

[0062] In some embodiments, the server may also be implemented as a terminal.

[0063] Please see details Figure 1a In the method provided in this application embodiment, feature extraction processing is performed on the container image to obtain a first number of (i.e., Figure 1a The k embedded feature sequences shown are given, wherein the container image includes a component to be detected located within the container; the first number of embedded feature sequences are subjected to a predetermined number of consecutive preset processing steps to obtain a second number of (i.e., Figure 1aThe j-th processing result features are shown, where the second number is less than the first number; attention mechanism processing is performed on each of the processing result features to obtain an attention result; fully connected processing is performed on each of the attention results to obtain a fully connected processing result; upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result map, which includes the component to be detected; deformation detection is performed on the component to be detected to obtain a deformation detection result of the component to be detected.

[0064] The above method performs a series of image recognition processes on a container image to obtain a component recognition result image including the component to be detected. Then, deformation detection is performed on the component to be detected included in the component recognition result image to obtain the deformation detection result of the component to be detected. Since the component to be detected is identified first, and the accurate shape of the component to be detected is obtained before detecting whether the component to be detected is deformed, the embodiments of this application can improve the accuracy of deformation detection of the component to be detected.

[0065] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0066] In this embodiment, a component deformation detection method is provided, such as... Figure 1b As shown, this component deformation detection method is applied to electronic devices, and the specific process of this method may include the following steps 110 to 160:

[0067] 110. Perform feature extraction processing on the container image to obtain a first number of embedded feature sequences, wherein the container image includes the component to be detected located in the container.

[0068] A container image is an image reflecting the state of container components, with the container as the main subject. Container images can be acquired by security camera equipment or transmitted from other electronic devices to the electronic devices performing steps 110 to 160. It should be understood that the method of acquiring container images should not be construed as a limitation of this application. Container components include corrugated panels, door lock bars, door handles, corner posts, corner fittings, bottom crossbeams, and flooring, etc. The component to be inspected can be any one of the aforementioned container components. The first quantity is a positive integer, and the specific value of the first quantity should not be construed as a limitation of this application.

[0069] Optionally, in one specific embodiment, the specific process of "performing feature extraction processing on the container image to obtain a first number of embedded feature sequences" may include the following steps 111 to 112:

[0070] 111. The container image is segmented to obtain a first number of segmentation results.

[0071] Segmentation of container images can be achieved through convolution. Each convolution operation yields a segmentation result. A first number of convolution operations can be performed on the container image to obtain a first number of convolution results, which in turn yields a first number of segmentation results.

[0072] Optionally, let's assume the convolution kernel is 7*7 and the stride is 4. Then, by using the 7*7 convolution kernel and the stride of 4 to perform convolution processing on the image to be detected, we can obtain multiple convolution results. Let's assume the number of convolution results is the first number. Then, the first number of convolution results are the first number of segmentation results.

[0073] Please see details Figure 1c Let's assume the container image is an 8-pixel by 8-pixel image. Figure 1c Each square 'b' in the diagram represents a pixel. Figure 1c If the box c in the diagram has a 7x7 convolution kernel, then with a stride of 4, the following can be performed: Figure 1c The convolution operation shown in (1) is as follows: Figure 1c The convolution operation shown in (2) is as follows: Figure 1c The convolution operation shown in (3) is as follows: Figure 1c The convolution operation is shown in (4) above. The above process involves four convolution operations, resulting in four convolution results, which are also four segmentation results; that is: the first quantity in Figure 1c In the specific implementation shown, the value is 4. It should be understood that... Figure 1c This is merely a specific example of segmenting a container image, and the dimensions of the container image, the size of the convolution kernel, and the stride should not be construed as limiting this application.

[0074] 112. Perform a linear transformation on each of the segmentation results to obtain the corresponding embedded feature sequence.

[0075] Optionally, each segmentation result can be straightened, and a linear embedding layer can be used to perform a linear transformation on the straightened segmentation result to obtain an embedded feature sequence.

[0076] In the above implementation, convolution processing can be used to segment the container image, obtaining a first number of segmentation results. For each segmentation result, the following identical operation can be performed: the segmentation result is straightened, and a linear embedding layer is used to perform a linear transformation on the straightened segmentation result to obtain the corresponding embedded feature sequence, resulting in a first number of embedded feature sequences. The linear embedding layer can perform a linear transformation on the sparse matrix (i.e., the straightened segmentation result) to obtain a dense matrix (i.e., the embedded feature sequence), thereby saving storage space and improving processing efficiency.

[0077] 120. Perform a preset number of consecutive processing steps on the first number of embedded feature sequences to obtain a second number of processing result features, wherein the second number is less than the first number.

[0078] The set quantity is a pre-defined value, which is a positive integer. The specific value of the set quantity should not be construed as a limitation on this application. The preset processing is a pre-defined processing operation, and the specific process of the preset processing will be described in detail below.

[0079] Optionally, in one specific embodiment, the set quantity may be set to 2; then, accordingly, step 120 may specifically include the following steps 121 to 122:

[0080] 121. Perform the preset processing on the first number of embedded feature sequences to obtain a third number of intermediate result features, wherein the third number is less than the first number and the third number is greater than the second number.

[0081] In step 121, the first number of embedded feature sequences can be subjected to a first preset processing to obtain a third number of intermediate result features. The third number is also a positive integer, and its value is between the first number and the second number.

[0082] Optionally, in one specific embodiment, the first preset processing performed in step 121 may specifically include the following steps 1211 to 1213:

[0083] 1211. Perform attention mechanism processing on each of the embedded feature sequences to obtain attention sub-results.

[0084] For ease of description, let's take the first embedded feature sequence as an example. Here, the first embedded feature sequence is any one of the first number of embedded feature sequences.

[0085] Obtain the query vector Q, keyword K, and feature vector V of the first embedded feature sequence.

[0086] Reduce the number of keywords K and the number of feature vectors V in the first embedded feature sequence according to a proportionality coefficient. Let the proportionality coefficient be denoted as ratio, where 0 < ratio < 1. The specific value of ratio can be obtained by the staff through experiments or selected by the staff through comprehensive comparison. The specific value of ratio should not be construed as a limitation of this application.

[0087] Let the original matrix size of keyword K in the first embedded feature sequence be [N, d], and the original matrix size of feature vector V in the first embedded feature sequence be [N, d]. Here, N is the number of first embedded features, and d is the dimension of each first embedded feature. Use the proportionality coefficient ratio to perform dimensionality reduction on the original matrix [N, d] of keyword K. The size of the matrix after dimensionality reduction is [N * ratio, d]. Let the keyword after dimensionality reduction be denoted as K'. Use the proportionality coefficient ratio to perform dimensionality reduction on the original matrix [N, d] of feature vector V. The size of the matrix after dimensionality reduction is [N * ratio, d]. Let the feature vector after dimensionality reduction be denoted as V'. The above dimensionality reduction process can greatly reduce the computational complexity of the subsequent calculation process, thereby improving the processing efficiency of the attention mechanism.

[0088] Optionally, the above dimensionality reduction process can be implemented through an average pooling operation. The size of the pooling kernel of the average pooling layer can be set by the staff according to the value of the proportionality coefficient ratio.

[0089] After calculating the keyword K' after dimensionality reduction of the first embedded feature sequence and the feature vector V' after dimensionality reduction of the first embedded feature sequence, calculate the attention results of Q, K', and V' in each of the h spaces, where h is a positive integer.

[0090] Optionally, in a specific implementation, calculating the attention results of Q, K', and V' in each of the h spaces can be specifically implemented as follows: [[ID=​​​​​​​​​​​​​​​​​​​​​In the i-th space, Q is obtained through W i Q Perform fully connected processing to obtain the fully connected processing result QW i Q ; For K' through W i K Perform a fully connected operation to obtain the fully connected result K'W. i K ; for V' through W i V Perform a fully connected operation to obtain the fully connected result V'W. i V .

[0093] The attention mechanism is applied to the three fully connected processing results to obtain the attention result corresponding to the i-th space:

[0094]

[0095] Among them, d in equation (1) head Let be the dimension of the i-th space.

[0096] For each of the h spaces, the attention result for each space can be obtained through the above steps, resulting in a total of h attention results: head1, head2, ... head1. h .

[0097] Given h attention results: head1, head2, ... head h By concatenating the components, we obtain the following concatenated result: [head1; head2; ... head] h ].

[0098] The above concatenation result is processed by a fully connected layer to obtain the attention sub-result corresponding to the first embedded feature sequence:

[0099] MH(Q,K',V')=[head1;head2;…head h W O .

[0100] MH(Q,K',V') represents context-related features. Each row of MH(Q,K',V') represents the contextual relationship between each embedded feature in the first embedded feature sequence and other embedded features. For example, the x-th row of MH(Q,K',V') represents the contextual relationship between the x-th embedded feature in the first embedded feature sequence and other embedded features in the first embedded feature sequence except for the x-th embedded feature. Fully connected layer W OThe parameters can be obtained by training the component recognition model. The attention mechanism described above can effectively extract contextual information by calculating the correlation between pairs of embedded features in the first embedded feature sequence.

[0101] Each embedded feature sequence can be processed using the attention mechanism described above, thereby obtaining the attention sub-result corresponding to each embedded feature sequence.

[0102] Optionally, in one specific implementation, the attention mechanism processing steps described above can be implemented in a multi-head self-attention module.

[0103] 1212. Perform fully connected processing on each attention sub-result to obtain a fully connected processing sub-result.

[0104] Optionally, for each attention sub-result, the attention sub-result can be input into two fully connected layers in series for fully connected processing, thereby obtaining a fully connected processed sub-result.

[0105] Let's assume that the two fully connected layers corresponding to the fully connected processing in step 1212 are W1, W2, W3, W4, W5, W6, W7, W8, W9, W1 ... F and W2 F W1 F and W2 F The parameters can be obtained by training the component recognition model. Continuing with the example above, taking the attention sub-result MH(Q,K',V') corresponding to the first embedded feature sequence as an example, the fully connected processing sub-result corresponding to the first embedded feature sequence is:

[0106] Fc1=[MH(Q,K',V')W1 F W2 F

[0107] It should be understood that the number of fully connected layers corresponding to fully connected processing can be two or other numbers, such as three. The specific number of fully connected layers should not be construed as a limitation of this application.

[0108] Optionally, in one specific implementation, the fully connected processing steps described above can be implemented in a feedforward neural network (FNN) module.

[0109] 1213. The first number of fully connected sub-results are downsampled to obtain the third number of intermediate result features.

[0110] By performing steps 1211 to 1212 on each of the first number of embedded feature sequences, first number of fully connected processing sub-results are obtained.

[0111] Optionally, the first number of fully connected processing sub-results can be processed by non-convolutional downsampling to obtain a third number of intermediate result features.

[0112] In the above embodiment, the above non-convolutional downsampling processing can be implemented in a Patch Merging layer. The Patch Merging layer can fuse different embedded feature sequences and reduce the number of embedded feature sequences, thereby facilitating the acquisition of a higher receptive field and higher-level semantic information.

[0113] 122. Perform the preset processing on the third number of intermediate result features to obtain the second number of processed result features.

[0114] In step 122, a second preset processing can be performed on the third number of intermediate result features, so that the second number of processed result features can be obtained. The value of the second number is less than the third number.

[0115] Optionally, in a specific embodiment, the first preset processing performed in step 122 may specifically include the following steps 1221 to 1223:

[0116] 1221. Perform an attention mechanism processing on each intermediate result feature to obtain a second attention sub-result.

[0117] For ease of description, hereinafter, the first intermediate result feature will be taken as an example for illustration. Among them, the first intermediate result feature is any one of the third number of intermediate result features.

[0118] Obtain the query vector Q1, key K1, and feature vector V1 of the first intermediate result feature.

[0119] Reduce the number of keys K1 and the number of feature vectors V1 of the first intermediate result feature according to a second proportionality coefficient. Let the second proportionality coefficient be denoted by ratio`, where 0 < ratio` < 1; specifically, the value of ratio` can be obtained by the staff through experiments or selected by the staff through comprehensive comparison. The specific value of ratio` should not be construed as a limitation of the present application.

[0120] Let the original matrix size of the keyword K1 of the first intermediate result feature be [N1, d1], and the original matrix size of the feature vector V1 of the first intermediate result feature be [N1, d1]. Here, N1 is the number of first intermediate result features, and d1 is the dimension of each first intermediate result feature. The original matrix [N1, d1] of the keyword K1 is reduced in dimension using the second scaling factor ratio`, resulting in a matrix size of [N1 * ratio`, d1]. Let's denote the dimension-reduced keyword as K1'. Similarly, the original matrix [N1, d1] of the feature vector V1 is reduced in dimension using the second scaling factor ratio`, resulting in a matrix size of [N1 * ratio`, d1]. Let's denote the dimension-reduced feature vector as V1'. This dimension reduction significantly reduces the computational load in subsequent calculations, thereby improving the processing efficiency of the attention mechanism.

[0121] Optionally, the dimensionality reduction process described above can be achieved through average pooling, where the size of the pooling kernel of the average pooling layer can be set by the operator based on the value of the second proportional coefficient ratio.

[0122] After calculating the dimension-reduced keyword K1' of the first intermediate result feature and the dimension-reduced feature vector V1' of the first embedded feature sequence, the attention results of Q1, K1', and V1' in each of the h1 spaces are calculated, where h1 is a positive integer.

[0123] Optionally, in one specific implementation, the attention results of Q1, K1', and V1' in each of the h1 spaces are calculated, which can be achieved in the following way:

[0124] Taking the i1th space out of h1 spaces as an example, These are the three fully connected layers in the i1th space. The parameters can be obtained by training the component recognition model.

[0125] In the i1th space, Q1 is passed through Perform fully connected processing to obtain the fully connected processing result. For K1' Perform a fully connected operation to obtain the fully connected result K1'. For V1' Perform a fully connected operation to obtain the fully connected result V1'.

[0126] The above three fully connected processing results are processed using an attention mechanism to obtain the attention result corresponding to the i1th space:

[0127]

[0128] Among them, d in equation (2) head1 Let i be the dimension of the i1th space.

[0129] For each of the h1 attention spaces, the attention result for each space can be obtained through the above steps, resulting in a total of h1 attention results: head1, head2, ...

[0130] Generate h1 attention results: head1, head2, ... By concatenating the elements, we obtain the following concatenated result: [head1; head2; ... ].

[0131] The above splicing result is processed by a fully connected layer to obtain the second attention sub-result corresponding to the first intermediate result feature:

[0132] MH(Q1,K1',V1')=[head1;head2;…head h1 W1 O .

[0133] MH(Q1,K1',V1') represents the contextual relationship features among the features included within the first intermediate result feature. Fully connected layer W1 O The parameters can be obtained by training the component recognition model. The above attention mechanism process can effectively extract contextual information by calculating the correlation between features included in the first intermediate result features.

[0134] Each intermediate result feature can be processed using the attention mechanism described above to obtain the second attention sub-result corresponding to each intermediate result feature.

[0135] Optionally, in one specific implementation, the attention mechanism processing steps described above can be implemented in a multi-head self-attention module.

[0136] 1222. Perform a fully connected operation on each second attention sub-result to obtain the second fully connected sub-result.

[0137] Optionally, for each second attention sub-result, the second attention sub-result can be input into two fully connected layers in series for fully connected processing, thereby obtaining the second fully connected processing sub-result.

[0138] Let's assume that the two fully connected layers corresponding to the fully connected processing in step 1222 are W1' and W1' respectively. Fand W2' F W1' F and W2' F The parameters can be obtained by training the component recognition model. Continuing with the example above, taking the second attention sub-result MH(Q1,K1',V1') corresponding to the first intermediate result feature as an example, the second fully connected processing sub-result corresponding to the first intermediate result feature is:

[0139] Fc2=[MH(Q1,K1',V1')W1' F W2' F

[0140] It should be understood that the number of fully connected layers corresponding to fully connected processing can be two or other numbers, such as three. The specific number of fully connected layers should not be construed as a limitation of this application.

[0141] Optionally, in one specific implementation, the fully connected processing steps described above can be implemented in a feedforward neural network (FNN) module.

[0142] 1223. Perform downsampling on the third number of second fully connected sub-results to obtain the second number of processing result features.

[0143] By performing steps 1221 to 1222 on each of the third number of intermediate result features, a second number of processing result features are obtained.

[0144] Optionally, a third number of second fully connected processing sub-results can be subjected to non-convolution downsampling processing to obtain a second number of processing result features.

[0145] In the above implementation, the non-convolutional downsampling process can be implemented in the Patch Merging layer. The Patch Merging layer can fuse different intermediate result features, thereby further reducing the number of embedded feature sequences and facilitating the acquisition of a higher receptive field and higher-level semantic information.

[0146] 130. Perform attention mechanism processing on each of the processing result features to obtain the attention result.

[0147] For ease of description, let's take the first processing result feature as an example. Here, the first processing result feature is any one of the second number of processing result features.

[0148] Obtain the query vector Q2, keyword K2, and feature vector V2 of the first processing result features.

[0149] Reduce the number of keywords K2 and the number of feature vectors V2 of the first processing result feature according to the third proportionality coefficient. Let's assume the third proportionality coefficient is represented by ratio``, where 0 < ratio`` < 1. Here, the specific value of ratio`` can be obtained by the staff through experiments or selected by the staff through comprehensive comparison. The specific value of ratio`` should not be construed as a limitation of this application.

[0150] Let's assume the original matrix size of keyword K2 of the first processing result feature is [N2, d2], and the original matrix size of feature vector V2 of the first processing result feature is [N2, d2]. Here, N2 is the number of the first processing result features, and d2 is the dimension of each first processing result feature. Use the third proportionality coefficient ratio`` to perform dimensionality reduction on the original matrix [N2, d2] of keyword K2. The size of the matrix after dimensionality reduction is [N2 * ratio``, d2]. Let's denote the keyword after dimensionality reduction as K2'. Use the third proportionality coefficient ratio`` to perform dimensionality reduction on the original matrix [N2, d2] of feature vector V2. The size of the matrix after dimensionality reduction is [N2 * ratio``, d2]. Let's denote the feature vector after dimensionality reduction as V2'. The above-mentioned dimensionality reduction process can greatly reduce the computational amount of the subsequent calculation process, thereby improving the processing efficiency of the attention mechanism.

[0151] Optionally, the above-mentioned dimensionality reduction process can be implemented through an average pooling operation. The size of the pooling kernel of this average pooling layer can be set by the staff according to the value of the third proportionality coefficient ratio``.

[0152] After calculating the keyword K2' after dimensionality reduction of the first processing result feature and the feature vector V2' after dimensionality reduction of the first embedding feature sequence, calculate the attention results of Q2, K2', and V2' in each of the h2 spaces, where h2 is a positive integer.

[0153] Optionally, in a specific implementation manner, calculating the attention results of Q2, K2', and V2' in each of the h2 spaces can be specifically implemented in the following way:

[0154] Taking the i2-th space among the h2 spaces as an example, are three fully connected layers in the i2-th space. The parameters of can be obtained by training the component recognition model.

[0155] In the i2-th space, perform a fully connected processing on Q2 through to obtain the fully connected processing result For K2' Perform a fully connected operation to obtain the fully connected result K2'. For V2' Perform a fully connected operation to obtain the fully connected result V2'.

[0156] The above three fully connected processing results are processed using an attention mechanism to obtain the attention result corresponding to the i2th space:

[0157]

[0158] Among them, equation (3) Let i be the dimension of the i2th space.

[0159] For each of the h² spaces, the attention result for each space can be obtained through the above steps, resulting in a total of h² attention results: head1, head2, ...

[0160] Generate h1 attention results: head1, head2, ... By concatenating the elements, we obtain the following concatenated result: [head1; head2; ... ].

[0161] The above concatenation result is processed using a fully connected layer to obtain the attention result corresponding to the features of the first processing result:

[0162]

[0163] MH(Q2,K2',V2') represents the contextual relationship features between the result features included within the first processing result feature. Fully connected layer W2 O The parameters can be obtained by training the component recognition model. The above attention mechanism process can effectively extract contextual information by calculating the correlation between the result features included in the first processing result features.

[0164] Each processing result feature can be processed using the attention mechanism described in the steps above, thereby obtaining the attention result corresponding to each processing result feature.

[0165] Optionally, in one specific implementation, the attention mechanism processing steps described above can be implemented in a multi-head self-attention module.

[0166] 140. Perform fully connected processing on each attention result to obtain the fully connected processing result.

[0167] Optionally, for each attention result, the attention result can be input into two fully connected layers in series for fully connected processing to obtain the fully connected processing result.

[0168] Let's assume that the two fully connected layers corresponding to the fully connected processing in step 140 are W1” F and W2” F W1” F and W2” F The parameters can be obtained by training the component recognition model. Continuing with the example above, taking the attention result MH(Q2,K2',V2') corresponding to the first processing result feature as an example, the fully connected processing result corresponding to the first processing result feature is:

[0169] Fc3=[MH(Q2,K2',V2')W1” F W2” F

[0170] It should be understood that the number of fully connected layers corresponding to fully connected processing can be two or other numbers, such as three. The specific number of fully connected layers should not be construed as a limitation of this application.

[0171] Optionally, in one specific implementation, the fully connected processing steps described above can be implemented in a feedforward neural network (FNN) module.

[0172] 150. Upsample the second number of fully connected processing results to obtain a component identification result image, which includes the component to be detected.

[0173] Optionally, in one implementation, the component identification result image can display multiple components, including the component to be detected. Different types of components can be displayed using different colors. For details, please refer to [link to relevant documentation]. Figure 1d As shown in (2); Figure 1d The one shown (1) is Figure 1d The container image corresponding to (2) shown; Figure 1d The diagram shown (3) is a schematic diagram of deformation detection of the part to be tested. The deformation detection process will be described in detail below.

[0174] In the above implementation, the component recognition result image displays multiple components including: corrugated plate surface, door lock rod, door handle, corner post, corner piece, etc., and each of the above components has its own corresponding color. Let's assume the component to be detected is the door lock rod (i.e., Figure 1d(c) shown in (2). While displaying the component to be tested, other components are also displayed, which helps to more clearly delineate the boundary of the component to be tested, thereby improving the detection accuracy of the boundary of the component to be tested.

[0175] Alternatively, in another implementation, the component recognition result image may highlight only the component to be detected. That is, the component recognition result image can use two different colors to display the component to be detected and the background (excluding the component to be detected). For details, please refer to [link to relevant documentation]. Figure 1e As shown in (2); Figure 1e The one shown (1) is Figure 1e The container image corresponding to (2) is shown. It should be understood that the specific way the component identification result diagram shows the component to be detected should not be construed as a limitation of this application.

[0176] 160. Perform deformation detection on the component to be tested to obtain the deformation detection result of the component to be tested.

[0177] The deformation detection results include two types: the component under test has deformed, and the component under test has not deformed. The final deformation detection result of the component under test can be obtained by following the deformation detection steps.

[0178] Optionally, in one specific embodiment, deformation detection of the component to be tested may specifically include the following steps 161 to 164:

[0179] 161. Perform skeleton extraction processing on the component to be detected to obtain the extraction processing result.

[0180] The extraction result includes multiple pixels used to characterize the skeleton shape of the component to be detected.

[0181] Skeleton extraction refines a connected region to a width of one pixel. It is commonly used for feature extraction and target topological representation. There are various ways to implement skeleton extraction; for example, it can be achieved through erosion operations, or using the `Skeletonize()` or `medial_axis()` functions. The specific method used to implement skeleton extraction should not be construed as a limitation of this application.

[0182] In this embodiment of the application, skeleton extraction will be described using erosion operations as an example: For Figure 1e (2) shows the identification segmentation map of the component to be detected, which is subjected to multiple etching operations to obtain the following result: Figure 1e The extraction processing results are shown in (3).

[0183] 162. Perform clustering processing on the extraction results to obtain clustering results.

[0184] Optionally, in one implementation, the clustering process may specifically include the following steps:

[0185] Randomly select one pixel from the multiple pixels included in the extraction result, and draw a circle with that pixel as the center and a set radius value. Count the number of pixels located within the drawn circle that represent the skeleton shape of the part to be detected.

[0186] If the number of statistically analyzed pixels is less than the preset threshold, the clustering operation for the randomly selected pixel is stopped, and a new pixel is randomly selected from the multiple pixels to perform the above circle drawing step.

[0187] If the number of pixels counted is greater than the preset threshold, then the above circle-drawing steps are performed sequentially for each pixel in the circle until the number of pixels in the newly drawn circle is less than the preset threshold.

[0188] After traversing all pixels included in the extracted processing results, the current clustering operation ends.

[0189] Clustering the extracted results from the components to be detected can more accurately distinguish the boundaries of each component. It should be understood that if there is only one component to be detected, the clustering operation can be omitted.

[0190] 163. Based on the clustering results, generate the deformation reference baseline of the component to be detected.

[0191] For each component to be tested, the center point of the end of the component can be obtained, and the center points of the ends can be connected to obtain the deformation reference baseline of the component to be tested.

[0192] Continuing with the example above, for a long, narrow door lock bar, the center points at both ends can be obtained, and the line connecting the two center points can be used as a reference line for the deformation of the door lock bar. For details, please refer to... Figure 1d (3) in the middle, Figure 1d (3) shows the four deformation reference lines corresponding to the four door lock rods.

[0193] 164. Based on the relative positional relationship between the plurality of pixels and the deformation reference baseline, determine the deformation detection result of the component to be detected.

[0194] For each component to be detected, the distance values ​​from all pixels in the extraction processing result corresponding to that component to be detected to the deformation reference baseline can be calculated, resulting in multiple distance values.

[0195] Calculate the standard deviation of the above distance values.

[0196] If the standard deviation is greater than the preset standard deviation threshold, or the number of distance values ​​greater than the preset distance threshold exceeds the preset number value, then the deformation detection result of the component to be tested is determined to be: the component to be tested has deformed.

[0197] If the standard deviation is not greater than the preset standard deviation threshold, and the number of distance values ​​greater than the preset distance threshold does not exceed the preset number value, then the deformation detection result of the component to be tested is determined to be: the component to be tested has not sent deformation.

[0198] For example, let's assume the extraction result corresponding to the component to be detected includes a total of g pixels. Then, we can calculate g distance values. We calculate the standard deviation z of these g distance values. If the standard deviation z is greater than a preset standard deviation threshold z', or the number p of distance values ​​greater than a preset distance threshold y exceeds a preset quantity p', then the deformation detection result of the component to be detected is determined to be: the component to be detected has deformed. If the standard deviation z is not greater than the preset standard deviation threshold z', and the number p of distance values ​​greater than the preset distance threshold y does not exceed the preset quantity p', then the deformation detection result of the component to be detected is determined to be: the component to be detected has not deformed.

[0199] In the above implementation, based on skeleton extraction, clustering algorithms, and the combined use of two metrics—standard deviation and the distance between a pixel and the deformation reference line—deformation detection of the component under test is comprehensively achieved, thus realizing the deformation detection function simply and effectively. For components with gentle deformation trends or small bending angles, the accuracy of deformation detection is further improved.

[0200] In one embodiment, the preset standard deviation threshold z' can be 3, and the preset quantity value p' can be 20. It should be understood that the specific values ​​of the preset standard deviation threshold z' and the preset quantity value p' should not be construed as limitations on this application.

[0201] Optionally, in one specific embodiment, before step 110, the embodiments of this application may further include the following steps A1 to A2:

[0202] A1. Obtain the component recognition model.

[0203] The component identification model can be downloaded by the electronic device from the server or obtained by the electronic device from other electronic devices. The specific method of obtaining the component identification model should not be construed as a limitation of this application.

[0204] A2. Train the component recognition model to obtain the trained component recognition model.

[0205] Among them, steps 110 to 150 can all be executed by the trained component recognition model.

[0206] Optionally, in one specific embodiment, step A2 may specifically include the following steps A21 to A27:

[0207] A21. Perform feature extraction processing on the training detection image to obtain a first number of training embedding feature sequences, wherein the training detection image includes a training detection component.

[0208] A22. Perform a preset number of consecutive processing steps on the first number of training embedded feature sequences to obtain a second number of training processing result features, wherein the second number is less than the first number.

[0209] A23. Perform attention mechanism processing on each of the training result features to obtain the training attention result.

[0210] A24. Perform fully connected processing on each of the training attention results to obtain the fully connected training processing result.

[0211] A25. Upsample the second number of training fully connected processing results to obtain a training component recognition result image, which includes the shape of the training detection component.

[0212] It should be understood that steps A21 to A25 correspond to the same processing procedures as steps 110 to 150 above, and will not be described in detail here.

[0213] A26. Based on the shape of the labeled detection component, the shape of the trained detection component, the number of samples in the trained detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample, a loss function is constructed.

[0214] The labeled detection component is the component to be identified, which is labeled by the user. The shape of the labeled detection component is the actual shape of the component to be identified.

[0215] The training detection part is the part to be identified by the part recognition model during training, based on the training detection image. The shape of the training detection part is the shape output by the part recognition model during training. Typically, there is an error between the shape of the training detection part and the shape of the labeled detection part.

[0216] The number of samples in the training detection image is the actual number of parts to be identified in the training detection image. Let's assume the part to be identified is a door lock bar. Figure 1d In (1), which is the training detection image, we can know the number of samples in the training detection image (i.e., Figure 1d The number of door lock bars shown in (1) is 4.

[0217] The recognition result category is the target recognition category of the component recognition model during training. For example, let's assume that the target of the component recognition model during training is to recognize four different categories of components: door lock bar, door handle, corner post, and corner piece. Then the recognition result categories would be door lock bar, door handle, corner post, and corner piece, and the number of recognition result categories would be 4.

[0218] The component recognition model can predict the probability value of each sample for each of the multiple recognition result categories. The predicted probability value of each sample for each recognition result category is denoted as the predicted probability of each sample belonging to each category. Continuing with the example above, let's take the four recognition result categories as door lock rod, door handle, corner post, and corner piece. For each sample, we can calculate the predicted probability that the sample belongs to the door lock rod, the predicted probability that the sample belongs to the door handle, the predicted probability that the sample belongs to the corner post, and the predicted probability that the sample belongs to the corner piece.

[0219] Each sample corresponds to a label category that is the actual category of the part to be identified as labeled by the user.

[0220] Optionally, in one specific embodiment, step A26 may specifically include the following steps A261 to A263:

[0221] A261. Based on the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample, a first loss function is constructed.

[0222] Specifically, the first loss function Loss1 mentioned above can be expressed by the formula:

[0223]

[0224] Where N is the number of samples in the training detection images; M is the number of categories in the recognition results; y ic Used to reflect the labeled category corresponding to sample i. If the actual category of sample i is equal to c, then y ic Select 1, otherwise y ic Take 0; p ic Let be the predicted probability that sample i belongs to category c.

[0225] A262. Based on the shape of the labeled detection component and the shape of the trained detection component, construct a second loss function.

[0226] Specifically, the second loss function Loss2 mentioned above can be expressed by the formula:

[0227]

[0228] Where X represents the two-dimensional matrix corresponding to the training detection component, which is used to reflect the shape of the training detection component; Y represents the two-dimensional matrix corresponding to the annotation detection component, which is used to reflect the shape of the annotation detection component.

[0229] A263. Based on the first loss function and the second loss function, construct the loss function.

[0230] Optionally, a sum of the first loss function and the second loss function can be constructed, and this sum can be used as the final loss function.

[0231] Specifically, the loss function can be represented by the following expression:

[0232] Loss = Loss1 + Loss2

[0233] A27. If the loss function meets the preset requirements, then the training of the component recognition model is completed, and the trained component recognition model is obtained.

[0234] Optionally, in one specific implementation, the loss function satisfies a preset requirement, which may specifically be:

[0235] The loss function Loss = Loss1 + Loss2 converges.

[0236] If the loss function Loss = Loss1 + Loss2 converges, it can be determined that the training of the component recognition model is complete, and the trained component recognition model is obtained.

[0237] In the above embodiments, the first loss function Loss1 is cross-entropy loss, and the second loss function is dice loss. Combining cross-entropy loss and dice loss when constructing the loss function can further improve the recognition accuracy of parts and obtain more accurate recognition results. Compared with traditional image technology, the method provided in this application embodiment can more robustly handle complex backgrounds and diverse part categories.

[0238] In the component deformation detection method provided in this application embodiment, feature extraction can be performed on a container image to obtain a first number of embedded feature sequences. Then, the first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processed result features. Each processed result feature is then subjected to attention mechanism processing and fully connected processing sequentially, resulting in a second number of fully connected processing results. Subsequently, upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result image including the component to be detected. Then, deformation detection is performed on the component to be detected included in the component recognition result image to obtain the deformation detection result of the component to be detected. In this application embodiment, for a container image, components included in the container image can be identified sequentially through feature extraction processing, predetermined number of processing steps, attention mechanism processing, fully connected processing, and upsampling processing, and the identification result includes the component to be detected. Since the component to be detected has been identified first, after obtaining the accurate shape of the component to be detected, the deformation of the component to be detected is then detected.

[0239] The embodiments of this application can improve the accuracy of deformation detection of the component to be tested.

[0240] The method described in the above embodiments will be further described in detail below.

[0241] In this embodiment, the method of this application embodiment will be described in detail using the door lock rod as an example of the component to be detected.

[0242] like Figure 2 As shown, the specific process of a component deformation detection method is as follows:

[0243] 201. The container image is segmented to obtain a first number of segmentation results.

[0244] 202. Perform a linear transformation on each of the segmentation results to obtain the corresponding embedded feature sequence.

[0245] 203. Perform the preset processing on the first number of embedded feature sequences to obtain a third number of intermediate result features, wherein the third number is less than the first number and greater than the second number.

[0246] Optionally, in one specific embodiment, step 203 may specifically include the following steps 2031 to 2033:

[0247] 2031. Perform attention mechanism processing on each of the embedded feature sequences to obtain attention sub-results.

[0248] 2032. Perform fully connected processing on each of the attention sub-results to obtain fully connected processing sub-results.

[0249] 2033. The first number of fully connected sub-results are downsampled to obtain the third number of intermediate result features.

[0250] 204. Perform preset processing on the third number of intermediate result features to obtain the second number of processed result features.

[0251] 205. Perform attention mechanism processing on each of the processing result features to obtain the attention result.

[0252] 206. Perform fully connected processing on each attention result to obtain the fully connected processing result.

[0253] 207. Upsample the second number of fully connected processing results to obtain a component identification result image, which includes the component to be detected.

[0254] 208. Perform skeleton extraction processing on the component to be detected to obtain the extraction processing result.

[0255] The extraction result includes multiple pixels used to characterize the skeleton shape of the component to be detected.

[0256] 209. Perform clustering processing on the extracted results to obtain clustering results.

[0257] 210. Based on the clustering results, generate the deformation reference baseline of the component to be detected.

[0258] 211. Based on the relative positional relationship between the plurality of pixels and the deformation reference baseline, determine the deformation detection result of the component to be detected.

[0259] The specific execution process of steps 201 to 211 has been explained in detail above, and will not be repeated here.

[0260] In the component deformation detection method provided in this application embodiment, feature extraction can be performed on a container image to obtain a first number of embedded feature sequences. Then, the first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processed result features. Each processed result feature is then subjected to attention mechanism processing and fully connected processing sequentially, resulting in a second number of fully connected processing results. Subsequently, upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result image including the component to be detected. Then, deformation detection is performed on the component to be detected included in the component recognition result image to obtain the deformation detection result of the component to be detected. In this application embodiment, for a container image, components included in the container image can be identified sequentially through feature extraction processing, predetermined number of processing steps, attention mechanism processing, fully connected processing, and upsampling processing, and the identification result includes the component to be detected. Since the component to be detected has been identified first, after obtaining the accurate shape of the component to be detected, the deformation of the component to be detected is then detected.

[0261] The embodiments of this application can improve the accuracy of deformation detection of the component to be tested.

[0262] To better implement the above methods, this application also provides a component deformation detection device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer (PC); the server can be a single server or a server cluster composed of multiple servers. For example, in this embodiment, the method of this application will be described in detail using the example of the component deformation detection device being specifically integrated into a terminal of an electronic device or deployed in the cloud.

[0263] For example, such as Figure 3 As shown, the component deformation detection device may include:

[0264] The feature extraction unit 301 is used to perform feature extraction processing on the container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located in the container;

[0265] The preset processing unit 302 is used to perform a preset processing on the first number of embedded feature sequences a set number of times to obtain a second number of processing result features, wherein the second number is less than the first number;

[0266] Attention mechanism unit 303 is used to perform attention mechanism processing on each of the processing result features to obtain an attention result;

[0267] The fully connected processing unit 304 is used to perform fully connected processing on each attention result to obtain a fully connected processing result.

[0268] The upsampling processing unit 305 is used to upsample the second number of fully connected processing results to obtain a component identification result image, wherein the component identification result image includes the component to be detected;

[0269] The deformation detection unit 306 is used to perform deformation detection on the component to be detected and obtain the deformation detection result of the component to be detected.

[0270] In one embodiment, the feature extraction unit 301 includes:

[0271] A segmentation processing subunit is used to segment the container image to obtain a first number of segmentation results;

[0272] The linear transformation subunit is used to perform linear transformation processing on each of the segmentation results to obtain the corresponding embedded feature sequence.

[0273] In one embodiment, the preset processing unit 302 includes:

[0274] The intermediate result subunit is used to perform the preset processing on the first number of embedded feature sequences to obtain a third number of intermediate result features, wherein the third number is less than the first number and the third number is greater than the second number;

[0275] The processing result subunit is used to perform the preset processing on the third number of intermediate result features to obtain the second number of processing result features.

[0276] In one implementation, the intermediate result subunit includes:

[0277] The attention sub-unit is used to process each of the embedded feature sequences using an attention mechanism to obtain the attention sub-result;

[0278] A fully connected sub-unit is used to perform fully connected processing on each of the attention sub-results to obtain a fully connected processing sub-result;

[0279] The downsampling sub-unit is used to downsample the first number of fully connected processing sub-results to obtain the third number of intermediate result features.

[0280] In one embodiment, the deformation detection unit 306 includes:

[0281] An extraction processing subunit is used to perform skeleton extraction processing on the component to be detected to obtain an extraction processing result, wherein the extraction processing result includes multiple pixels used to characterize the skeleton shape of the component to be detected;

[0282] The clustering subunit is used to perform clustering processing on the extraction results to obtain clustering results;

[0283] The baseline subunit is used to generate a deformation reference baseline for the component to be detected based on the clustering results.

[0284] The deformation detection subunit is used to determine the deformation detection result of the component to be detected based on the relative positional relationship between the plurality of pixels and the deformation reference baseline.

[0285] In one embodiment, the device further includes:

[0286] The model acquisition unit is used to acquire the component recognition model;

[0287] The model training unit is used to train the component recognition model to obtain the trained component recognition model; wherein, the steps of performing feature extraction processing on the container image to obtain a first number of embedded feature sequences, and the steps of upsampling processing on a second number of fully connected processing results to obtain a component recognition result image are executed by the trained component recognition model.

[0288] In one implementation, the model training unit includes:

[0289] The feature extraction training subunit is used to perform feature extraction processing on the training detection image to obtain a first number of training embedding feature sequences, wherein the training detection image includes a training detection component;

[0290] The training processing result subunit is used to continuously perform a preset number of processing operations on the first number of training embedded feature sequences to obtain a second number of training processing result features, wherein the second number is less than the first number.

[0291] A training attention subunit is used to process each of the training result features using an attention mechanism to obtain the training attention result.

[0292] A fully connected processing subunit is used to perform fully connected processing on each of the training attention results to obtain the training fully connected processing result;

[0293] An upsampling processing subunit is used to upsample the second number of training fully connected processing results to obtain a training component recognition result image, which includes the shape of the training detection component.

[0294] The loss function construction subunit is used to construct a loss function based on the shape of the labeled detection component, the shape of the training detection component, the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample.

[0295] The training decision subunit is used to determine that the component recognition model has been trained when the loss function meets the preset requirements, and to obtain the trained component recognition model.

[0296] In one implementation, training the decision subunit includes:

[0297] The first loss subunit is used to construct a first loss function based on the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the label category corresponding to each sample.

[0298] The second loss subunit is used to construct a second loss function based on the shape of the labeled detection component and the shape of the trained detection component;

[0299] The loss function construction sub-unit is used to construct the loss function based on the first loss function and the second loss function.

[0300] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0301] In the component deformation detection method provided in this application embodiment, feature extraction can be performed on a container image to obtain a first number of embedded feature sequences. Then, the first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processed result features. Each processed result feature is then subjected to attention mechanism processing and fully connected processing sequentially, resulting in a second number of fully connected processing results. Subsequently, upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result image including the component to be detected. Then, deformation detection is performed on the component to be detected included in the component recognition result image to obtain the deformation detection result of the component to be detected. In this application embodiment, for a container image, components included in the container image can be identified sequentially through feature extraction processing, predetermined number of processing steps, attention mechanism processing, fully connected processing, and upsampling processing, and the identification result includes the component to be detected. Since the component to be detected has been identified first, after obtaining the accurate shape of the component to be detected, the deformation of the component to be detected is then detected.

[0302] The embodiments of this application can improve the accuracy of deformation detection of the component to be tested.

[0303] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0304] In some embodiments, the component deformation detection device can also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the component deformation detection method of this application.

[0305] In this embodiment, the electronic device will be used as an example for detailed description, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0306] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0307] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0308] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0309] The electronic device also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0310] The electronic device may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0311] The electronic device may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0312] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0313] Feature extraction is performed on a container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located within the container; the first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processing result features, the second number being less than the first number; attention mechanism processing is applied to each of the processing result features to obtain an attention result; fully connected processing is applied to each of the attention results to obtain a fully connected processing result; upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result image, the component recognition result image including the component to be detected; deformation detection is performed on the component to be detected to obtain a deformation detection result of the component to be detected.

[0314] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0315] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0316] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the component deformation detection methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0317] Feature extraction is performed on a container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located within the container; the first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processing result features, the second number being less than the first number; attention mechanism processing is applied to each of the processing result features to obtain an attention result; fully connected processing is applied to each of the attention results to obtain a fully connected processing result; upsampling processing is performed on the second number of fully connected processing results to obtain a component recognition result image, the component recognition result image including the component to be detected; deformation detection is performed on the component to be detected to obtain a deformation detection result of the component to be detected.

[0318] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0319] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0320] Since the instructions stored in the storage medium can execute the steps in any of the component deformation detection methods provided in the embodiments of this application, the beneficial effects that any of the component deformation detection methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0321] The above provides a detailed description of a component deformation detection method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting component deformation, characterized in that, The method includes: The container image is processed by feature extraction to obtain a first number of embedded feature sequences, wherein the container image includes the component to be detected located in the container; The first number of embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of processed feature sequences, where the second number is less than the first number. The predetermined processing steps include dimensionality reduction, attention mechanism processing, fully connected processing, and downsampling processing. Each of the processing result features is processed using an attention mechanism to obtain an attention result; Each attention result is processed using a fully connected layer to obtain the fully connected processing result. Upsampling is performed on the second number of fully connected processing results to obtain a component identification result image, which includes the component to be detected. Deformation detection is performed on the component to be tested to obtain the deformation detection result of the component to be tested.

2. The method as described in claim 1, characterized in that, The feature extraction process on the container image yields a first number of embedded feature sequences, including: The container image is segmented to obtain a first number of segmentation results; A linear transformation is performed on each of the segmentation results to obtain the corresponding embedded feature sequence.

3. The method as described in claim 1, characterized in that, The step of performing a predetermined number of consecutive processing operations on the first number of embedded feature sequences to obtain a second number of processed result features includes: The first number of embedded feature sequences are subjected to the preset processing to obtain a third number of intermediate result features, wherein the third number is less than the first number and the third number is greater than the second number; The preset processing is performed on the third number of intermediate result features to obtain the second number of processed result features.

4. The method as described in claim 3, characterized in that, The step of performing the preset processing on the first number of embedded feature sequences to obtain a third number of intermediate result features includes: Each of the embedded feature sequences is processed using an attention mechanism to obtain an attention sub-result; Perform fully connected processing on each attention sub-result to obtain a fully connected processing sub-result; The first number of fully connected sub-results are downsampled to obtain the third number of intermediate result features.

5. The method as described in claim 1, characterized in that, The deformation detection of the component to be tested, and the obtaining of the deformation detection result of the component to be tested, includes: The skeleton of the component to be detected is extracted to obtain the extraction result, wherein the extraction result includes multiple pixels used to characterize the skeleton shape of the component to be detected. The extraction results are then subjected to clustering to obtain clustering results; Based on the clustering results, a deformation reference baseline for the component to be detected is generated; Based on the relative positional relationship between the plurality of pixels and the deformation reference baseline, the deformation detection result of the component to be detected is determined.

6. The method as described in claim 1, characterized in that, Before performing feature extraction processing on the container image to obtain a first number of embedded feature sequences, the method further includes: Obtain the component recognition model; The component recognition model is trained to obtain a trained component recognition model; wherein, the steps of performing feature extraction processing on the container image to obtain a first number of embedded feature sequences, and the steps of upsampling processing on a second number of fully connected processing results to obtain a component recognition result image are executed by the trained component recognition model.

7. The method as described in claim 6, characterized in that, The step of training the component recognition model to obtain the trained component recognition model includes: The training detection image is subjected to feature extraction processing to obtain a first number of training embedding feature sequences, wherein the training detection image includes a training detection component; The first number of training embedded feature sequences are subjected to a predetermined number of consecutive processing steps to obtain a second number of training processing result features, where the second number is less than the first number. Each of the training result features is processed using an attention mechanism to obtain the training attention result; Each of the training attention results is processed using a fully connected network to obtain the training fully connected network processing result. Upsampling is performed on a second number of the training fully connected processing results to obtain a training component recognition result image, which includes the shape of the training detection component. A loss function is constructed based on the shape of the labeled detection component, the shape of the trained detection component, the number of samples in the trained detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample. If the loss function meets the preset requirements, the training of the component recognition model is determined to be complete, and the trained component recognition model is obtained.

8. The method as described in claim 7, characterized in that, The loss function is constructed based on the shape of the labeled detection component, the shape of the trained detection component, the number of samples in the trained detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the labeled category corresponding to each sample, including: Based on the number of samples in the training detection image, the number of recognition result categories, the predicted probability of each sample belonging to each category, and the label category corresponding to each sample, a first loss function is constructed. A second loss function is constructed based on the shape of the labeled detection component and the shape of the trained detection component. The loss function is constructed based on the first loss function and the second loss function.

9. A component deformation detection device, characterized in that, The device includes: A feature extraction unit is used to perform feature extraction processing on a container image to obtain a first number of embedded feature sequences, wherein the container image includes a component to be detected located in the container; A preset processing unit is used to continuously perform a preset processing on the first number of embedded feature sequences a set number of times to obtain a second number of processed result features, wherein the second number is less than the first number, and the preset processing includes dimensionality reduction processing, attention mechanism processing, fully connected processing, and downsampling processing. An attention mechanism unit is used to perform attention mechanism processing on each of the processing result features to obtain an attention result; A fully connected processing unit is used to perform fully connected processing on each attention result to obtain a fully connected processing result; An upsampling processing unit is used to upsample the second number of fully connected processing results to obtain a component identification result image, wherein the component identification result image includes the component to be detected; The deformation detection unit is used to perform deformation detection on the component to be tested and obtain the deformation detection result of the component to be tested.

10. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps in the component deformation detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the steps in the component deformation detection method according to any one of claims 1 to 8.

Images