Sole and vamp material placement angle detection method, device and equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-04
- Publication Date
- 2026-04-10
Smart Images
Figure CN121837582A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of machine vision and automation, and in particular to a method, apparatus, device, and storage medium for detecting the placement angle of shoe sole and upper materials. Background Technology
[0002] Currently, with the continuous enrichment of new technologies, footwear production technology and equipment are constantly being upgraded, and artificial intelligence is accelerating its integration with traditional manufacturing. In the shoe production process, multiple steps are required. On the production line, the orientation of the incoming sole and upper materials is often disordered, which is detrimental to subsequent processes such as adhesive spraying, cleaning agent spraying, and other procedures. Therefore, whether for adhesive spraying or cleaning, a target detection system capable of quickly identifying the sole's orientation is essential.
[0003] While existing object detection algorithms have achieved excellent results in remote sensing images and natural scene text, research on object detection algorithms using deep learning technology in the footwear manufacturing industry is relatively limited. Currently, mainstream object detection algorithms delineate the object's outline using horizontal bounding boxes. However, when it's necessary to obtain the object's angle, simply delineating the object's outline using horizontal boxes is insufficient. Summary of the Invention
[0004] In order to detect the placement angle of an object, this application provides a method, apparatus, device, and storage medium for detecting the placement angle of shoe sole and upper materials.
[0005] The above-mentioned objective of this application is achieved through the following technical solution: A method for detecting the placement angle of shoe sole and upper materials, wherein the method for detecting the posture of shoe sole and upper materials includes: Images of shoes with different types of sole and upper materials are collected, and the images are augmented. Based on the data augmentation results, the features of the image are labeled using a rotated target detection box to obtain a rotated labeled box. The labeled target images are then used to construct a training set and a validation set according to a preset division ratio. Construct the total loss function using Wasserstein distance loss; The training set and the validation set are input into the YOLOv5 network model, and the YOLOv5 network model is trained by improving it according to the total loss function to obtain the detection model for the placement angle of the sole and upper materials. A test image containing the object to be tested is acquired, and the test image is input into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle.
[0006] By adopting the above technical solution, images of different types of shoe sole and upper materials are acquired by taking top-down photos. To improve the detection and generalization capabilities of the model, data augmentation processing is performed on the images, resulting in a larger dataset for model training and more accurate training results. Image features are acquired and rotated object detection boxes are used to replace horizontal detection boxes for image feature annotation, enabling multi-angle recognition of image placement angles. Since the image features in the training set incorporate rotated object detection boxes and angles, the original loss function RIoU in the YOLOv5 network model has sparse non-differentiable points, making it unsuitable as a loss function for the rotated object detection network model. Therefore, Wasserstein distance loss is introduced to measure the similarity between the labeled boxes and the predicted boxes. Wasserstein distance loss is used to replace the bounding box loss to construct the total loss function, further improving the accuracy of detecting the placement angle of shoe sole and upper materials.
[0007] In a preferred embodiment, this application can be further configured as follows: the acquisition of images containing different types of sole and upper materials, and the data augmentation of the images, specifically include: Obtain the image enhancement processing type, which includes cropping, translation, brightness adjustment, noise addition, rotation, and flipping; One or more of the image enhancement processing types are randomly selected, and data enhancement processing is performed on each of the images.
[0008] By adopting the above technical solution, the acquired images are augmented to achieve the purpose of data expansion. The random effects of the augmentation type and the combined effect of multiple processing types increase the diversity and robustness of the image data. The augmented data provides a larger dataset for model training, reduces the risk of overfitting, and improves the training effect of the model.
[0009] In a preferred example, this application can be further configured as follows: the step of inputting the training set and the validation set into a YOLOv5 network model, and training the YOLOv5 network model by improving it according to the total loss function, to obtain a detection model for the placement angle of the sole and upper materials, specifically includes: Receive the training set and the validation set; The training set is trained according to the YOLOv5 network model, and the rotation prediction box of image features is obtained from the training process; The distance index between the rotated prediction box and the rotated label box is calculated based on the total loss function; The optimal rotation prediction box is selected based on the distance calculation results, and the optimal rotation prediction box is verified based on the verification set; the detection model of the placement angle of the sole and upper materials is obtained based on the verification results.
[0010] By adopting the above technical solution, the training set is trained based on the YOLOv5 network model to obtain the trained rotation prediction box. Since the loss function of the original YOLOv5 network model calculates the loss of the quality and distance of the horizontal detection box, the original loss function cannot calculate the angle loss after replacing the horizontal detection box with the rotation target detection box. Therefore, the Wasserstein distance loss is introduced to form a new total loss function to calculate the quality and angle loss of the rotation prediction box and the rotation annotation box. Based on the calculation results, the rotation prediction box with the minimum quality and angle loss is selected, and the optimal rotation prediction box is verified and adjusted using the validation set to obtain the optimal hyperparameters, thereby obtaining the optimal detection model for the placement angle of the sole and upper materials.
[0011] In a preferred embodiment, this application can be further configured such that the construction of the total loss function using Wasserstein distance loss specifically includes: The rotated target detection box is converted into a two-dimensional Gaussian distribution N(m,∑): m = (x, y) Where (x,y) represents the coordinates of the center point of the rotating target detection box, w and h represent the width and height of the rotating target detection box, respectively, θ represents the angle between the long side of the rotating target detection box and the horizontal rightward direction, R is the rotation matrix, S is the diagonal vector of the eigenvalues, and m is the mean of the Gaussian distribution. The rotated prediction box and the rotated label box are converted into their corresponding two-dimensional Gaussian distribution representations, and the Wasserstein distance between the two two-dimensional Gaussian distributions is calculated: The Wasserstein distance is transformed into an affinity metric τ+f(d) to represent the IoU between two bounding boxes using a nonlinear transformation f. 2 ), where f() represents a nonlinear function and τ represents a hyperparameter, the formula for calculating the Wasserstein distance loss is: Where d is the square root of the Wasserstein distance between the rotated prediction box and the rotated label box; Label classification loss L cls And confidence loss L conf The binary cross-entropy loss function is used: Where N is the total number of label categories, x i y is the predicted value for the current category. i This represents the probability of the current label category after the activation function. It is the actual value of the current tag category; The label classification loss, the confidence loss, and the Wasserstein distance loss constitute the total loss function, and the formula for calculating the total loss function (Loss) is as follows: Loss = L gwd +L cls +L conf , The YOLOv5-GWD model is constructed based on the total loss function Loss.
[0012] By adopting the above technical solution, the rotated bounding box and the rotated predicted bounding box are respectively converted into two-dimensional Gaussian distributions. Then, the Wasserstein distance is used to calculate the distance between the two Gaussian distributions, that is, the distance between the rotated bounding box and the rotated predicted bounding box. Since Wasserstein is very sensitive to large errors, a nonlinear transformation f is used to convert the Wasserstein distance into an affinity metric τ+f(d) similar to the IoU between two bounding boxes. 2 ), where f() represents a nonlinear function, with respect to the Wasserstein distance d 2 A transformation is performed to make the loss smoother and more expressive. When the predicted box and the label box completely overlap, the RIoU between the two boxes is 1. At this time, the predicted box and the label box will generate the same two-dimensional Gaussian distribution, that is, the Wasserstein distance between the two boxes is also equal to 0. As the predicted box and the label box gradually move away from each other, the Wasserstein distance between them also increases. Moreover, the Wasserstein distance includes the center point distance component of the two-dimensional Gaussian distribution. Therefore, the Wasserstein distance loss can still measure the distance between the two boxes when the predicted box and the label box do not intersect. It can be seen that the Gaussian Wasserstein distance loss is consistent with the RIoU-based evaluation metric, while also avoiding the problem of discontinuity in the rotation angle range and reducing the learning difficulty of the model.
[0013] In a preferred embodiment, this application can be further configured as follows: acquiring a test image carrying the object to be tested, inputting the test image into the shoe sole and upper material placement angle detection model, and obtaining the detection result of the object placement angle, specifically includes: The image to be tested will be processed in terms of type. A detection set is generated based on the processing results of the described type; The detection set is input into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle.
[0014] By adopting the above technical solution, the image to be tested is processed in terms of type, including cropping, translation, brightness change, noise addition, rotation, flipping, and type conversion, so that the data type of the input detection model is consistent with the data type during model training, so that the detection model can achieve the same detection accuracy for the placement angle of the image to be tested as the optimal detection accuracy during model training.
[0015] The second objective of this invention is achieved through the following technical solution: A device for detecting the placement angle of a shoe sole and upper, the device comprising: The image acquisition module is used to acquire images of different types of shoe sole and upper materials, and to perform data augmentation on the images; The image feature annotation module is used to annotate the features of the image using a rotated target detection box based on the data augmentation result, to obtain a rotated annotation box, and to construct a training set and a validation set for the annotated target images according to a preset division ratio; the total loss function construction module is used to construct a total loss function using Wasserstein distance loss; The model training module inputs the training set and the validation set into the YOLOv5 network model, and improves the YOLOv5 network model according to the total loss function to train it, thereby obtaining a detection model for the placement angle of the sole and upper materials. The real-time detection module acquires a test image carrying the object to be tested, inputs the test image into the shoe sole and upper material placement angle detection model, and obtains the detection result of the object placement angle.
[0016] The above-mentioned objective three of this application is achieved through the following technical solution: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for detecting the placement angle of shoe sole and upper materials.
[0017] The fourth objective of this application is achieved through the following technical solution: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for detecting the placement angle of shoe sole and upper materials.
[0018] In summary, this application includes at least one of the following beneficial technical effects: 1. Images of different types of shoe sole and upper materials are collected by taking top-down photos. In order to improve the detection and generalization capabilities of the model, data augmentation processing is performed on the images. The dataset used for model training is larger, and the training results are more accurate. 2. Image features are acquired and rotated object detection boxes are used to replace horizontal detection boxes to label image features, thereby enabling multi-angle recognition of image placement angles. Since the image features in the training set adopt rotated object detection boxes and the addition of angles, the calculation process of the original loss function RIOU in the YOLOv5 network model has sparse non-differentiable points, which makes it unsuitable as a loss function for the rotated object detection network model. Therefore, Wasserstein distance loss is introduced to measure the similarity between labeled boxes and predicted boxes. Wasserstein distance loss is used to replace bounding box loss to construct the total loss function, which further improves the accuracy of detecting the placement angle of shoe sole and upper materials. 3. Convert the rotated bounding box and the rotated predicted bounding box into two-dimensional Gaussian distributions, respectively. Then, use the Wasserstein distance to calculate the distance between the two Gaussian distributions, i.e., the distance between the rotated bounding box and the rotated predicted bounding box. Since Wasserstein is very sensitive to large errors, a nonlinear transformation f is used to convert the Wasserstein distance into an affinity metric τ+f(d) similar to the IoU between two bounding boxes. 2 ), where f() represents a nonlinear function, with respect to the Wasserstein distance d 2 A transformation is performed to make the loss smoother and more expressive. When the predicted box and the label box completely overlap, the RIoU between the two boxes is 1. At this time, the predicted box and the label box will generate the same two-dimensional Gaussian distribution, that is, the Wasserstein distance between the two boxes is also equal to 0. As the predicted box and the label box gradually move away from each other, the Wasserstein distance between them also increases. Moreover, the Wasserstein distance includes the center point distance component of the two-dimensional Gaussian distribution. Therefore, the Wasserstein distance loss can still measure the distance between the two boxes when the predicted box and the label box do not intersect. It can be seen that the Gaussian Wasserstein distance loss is consistent with the RIoU-based evaluation metric, while also avoiding the problem of discontinuity in the rotation angle range and reducing the learning difficulty of the model. Attached Figure Description
[0019] Figure 1 This is a flowchart of a method for detecting the placement angle of the sole and upper in an embodiment of this application; Figure 2 This is a diagram of an image acquisition device for a method of detecting the placement angle of the sole and upper in an embodiment of this application; Figure 3 This is a schematic diagram of a two-dimensional Gaussian distribution modeling rotating frame of a method for detecting the placement angle of the sole and upper in an embodiment of this application; Figure 4 This is a YOLOv5 network structure diagram of the method for detecting the placement angle of the sole and upper in an embodiment of this application; Figure 5 This is another flowchart of the method for detecting the placement angle of the sole and upper in one embodiment of this application; Figure 6This is a flowchart illustrating the implementation of step S10 of the method for detecting the placement angle of the sole and upper in an embodiment of this application. Figure 7 This is a flowchart illustrating the implementation of step S40 of the method for detecting the placement angle of the sole and upper in an embodiment of this application. Figure 8 This is a flowchart illustrating step S30 of the method for detecting the placement angle of the sole and upper in an embodiment of this application. Figure 9 This is a flowchart illustrating the implementation of step S50 of the method for detecting the placement angle of the sole and upper in an embodiment of this application. Figure 10 This is a schematic block diagram of a shoe sole and upper placement angle detection device according to an embodiment of this application; Figure 11 This is a schematic diagram of a device according to one embodiment of this application. Detailed Implementation
[0020] The present application will be further described in detail below with reference to the accompanying drawings.
[0021] In one embodiment, such as Figure 5 As shown, this application discloses a method for detecting the placement angle of shoe sole and upper materials, which specifically includes the following steps: S10: Collect images of different types of shoe sole and upper materials and perform data augmentation on the images.
[0022] In this embodiment, data augmentation refers to generating new training samples by performing a series of transformations and modifications on the original image, thereby expanding the size and diversity of the training set.
[0023] Specifically, different styles and types of shoe soles and various shoe upper materials were selected from the manufacturer. Images of these materials were captured using a top-down photographic method. The captured images were then subjected to data augmentation, such as... Figure 2 As shown, the camera lens takes a top-down view of the sole and upper material of the shoe. The distance between the camera lens and the object being measured is 770mm. Light sources are placed at equal distances on both sides of the same horizontal line as the camera.
[0024] S20: Based on the data augmentation results, the features of the image are labeled using a rotated target detection box to obtain a rotated labeled box. The labeled target images are then used to construct a training set and a validation set according to a preset division ratio.
[0025] In this embodiment, a rotated target detection box refers to a detection box that can detect the angular position of image features. A rotated annotation box refers to a detection box that has been annotated with the angular position information of image features. The training set refers to a portion of the acquired images used as the image data set for training the model. The validation set refers to a separate sample set from the acquired images, used to adjust the model's hyperparameters and to perform preliminary evaluation of the model's training results.
[0026] Specifically, a rotating target detection bounding box is used to label the upper material and sole of the shoe. The labeling data includes the label category of the rotating bounding box, the x and y coordinates of the center point, the width, the height and the rotation angle. A rotating annotation bounding box is generated based on the annotation data. The labeled target images are used to construct a training set and a validation set according to a preset division ratio. In one embodiment, the data volume division ratio between the training set and the validation set is 8:2.
[0027] S30: Construct the total loss function using Wasserstein distance loss.
[0028] In this embodiment, the Wasserstein distance loss is a measure of the distance between the rotated bounding box and the rotated predicted bounding box. The total loss function is a function used to calculate the loss of quality and angle of the rotated predicted bounding box in the training model.
[0029] Specifically, since the image features of the rotated target detection box are used and angle data is added, the original loss function applied to the horizontal prediction box in the YOLOv5 network model is no longer applicable. Therefore, the YOLOv5 network is improved by introducing the Wasserstein distance loss to form a new total loss function to measure the quality and angle loss between the rotated prediction box and the rotated annotation box.
[0030] S40: Input the training set and validation set into the YOLOv5 network model, and improve the YOLOv5 network model according to the total loss function to train it, so as to obtain the detection model of the placement angle of the sole and upper materials.
[0031] In this embodiment, the YOLOv5 network model refers to an object detection algorithm. The sole and upper material placement angle detection model refers to a model that can detect the placement angle of the sole and upper materials in real time.
[0032] Specifically, the YOLOv5 network model is improved by introducing Wasserstein distance loss. The training set is fed into the improved YOLOv5 network model for training. The similarity between the labeled box and the predicted box is measured by Wasserstein distance, and this is used as a metric to calculate the loss. The hyperparameters of the model are adjusted using the validation set and used to conduct a preliminary evaluation of the model training results. Finally, the optimal detection model for the placement angle of the sole and upper materials is obtained.
[0033] S50: Acquire the image of the object to be tested, input the image of the object to be tested into the detection model of the placement angle of the sole and upper material, and obtain the detection result of the placement angle of the object.
[0034] In this embodiment, the image to be tested refers to the image of the sole and upper material of the shoe at the angle to be detected.
[0035] Specifically, the image to be tested is obtained by taking a top-down picture of the sole and upper material of the shoe at the angle to be tested. The optimal sole and upper material placement angle detection model trained is used to detect the image to be tested in real time and obtain the detection result of the object placement angle in real time.
[0036] In one embodiment, such as Figure 6 As shown, in step S10, images of different types of shoe sole and upper materials are acquired, and data augmentation is performed on the images, specifically including: S11: Obtain the image enhancement processing type, which includes cropping, translation, brightness adjustment, noise addition, rotation, and flipping.
[0037] In this embodiment, the image enhancement processing type refers to a data augmentation method commonly used in object detection, which can generate new training images by combining multiple different enhancement methods applied to the image.
[0038] Specifically, image enhancement processing type refers to a data augmentation method commonly used in object detection. It can generate new training images by combining multiple different enhancement methods applied to an image. Image enhancement processing types include cropping, translation, brightness adjustment, noise addition, rotation, and flipping.
[0039] S12: Randomly select one or more image enhancement processing types and perform enhancement processing on each image.
[0040] Specifically, one or more image enhancement processing types are randomly selected, and data enhancement processing is performed on each image to generate new training images. In one embodiment, the original image (a) is subjected to brightness change + rotation + flip to obtain a new training image (b), the original image (a) is subjected to rotation + Gaussian noise to obtain a new training image (c), and the original image (a) is subjected to translation + cropping to obtain a new training image (d), so as to achieve the purpose of expanding the training data through data enhancement.
[0041] In one embodiment, such as Figure 7 As shown, in step S40, the training set and validation set are input into the YOLOv5 network model, and the YOLOv5 network model is trained according to the total loss function to obtain the detection model for the placement angle of the sole and upper materials. Specifically, this includes: S41: Receive the training set and validation set.
[0042] Specifically, the YOLOv5 network model receives training and validation sets at the input end, extracts image features from the training set, and then processes the image features through the Focus structure, CSP structure, and SPP module to enrich the image feature information.
[0043] S42: Train the training set based on the YOLOv5 network model and obtain the rotation prediction box of image features from the training process.
[0044] In this embodiment, the rotated prediction box refers to the image feature prediction result represented by the rotated box obtained by training the training set in the YOLOv5 network model.
[0045] Specifically, rotation prediction boxes for image features are obtained by training the training set based on the YOLOv5 network model. The YOLOv5 network model is as follows: Figure 4 As shown, the YOLOv5 network structure consists of four parts: the input, the backbone, the neck, and the prediction. The backbone is used to extract image features and is mainly composed of the Focus structure, the CSP structure, and the Spatial Pyramid Pooling (SPP) module. The Focus module increases the receptive field and fuses image features without increasing computation or losing resolution. The CSP module first divides the image features into two parts and then merges them through a cross-layer structure, reducing computation and ensuring accuracy. The SPP module connects different feature maps using different pooling methods, which can significantly separate contextual features, enabling the model to detect complex scenes. The neck fuses upsampled and downsampled feature maps, enriching the feature information. The prediction module predicts the final result using features at three different scales.
[0046] S43: Calculate the distance index between the rotated prediction box and the rotated label box based on the total loss function.
[0047] In this embodiment, the distance index refers to the quality and angular loss between the rotated prediction box and the rotated annotation box.
[0048] Specifically, the total loss function includes label classification loss, confidence loss, and Wasserstein distance loss. The label classification loss, confidence loss, and Wasserstein distance loss for the two rotated boxes are calculated separately. Finally, the calculation results are added together to obtain the quality and angle loss between the predicted box and the labeled box.
[0049] S44: Select the optimal rotation prediction box based on the distance calculation results, and validate the optimal rotation prediction box based on the validation set.
[0050] In this embodiment, the distance calculation result refers to the quality and angle loss between the predicted bounding box and the labeled bounding box calculated using the total loss formula. The optimal rotated predicted bounding box is the predicted bounding box with the highest similarity to the labeled bounding box.
[0051] Specifically, based on the calculation results of the quality and angle loss between the predicted box and the labeled box, the predicted box with the highest similarity to the labeled box is selected as the optimal rotation prediction box. The optimal rotation prediction box is then validated using the validation set, and the hyperparameters are adjusted.
[0052] S45: Based on the verification results, obtain the detection model for the placement angle of the sole and upper materials.
[0053] Specifically, the model training results are initially evaluated using a validation set. For example, when the predicted bounding box and the labeled bounding box completely overlap, the predicted bounding box and the ground truth bounding box will generate the same two-dimensional Gaussian distribution, that is, the Wasserstein distance between the two boxes is equal to 0. The hyperparameters are adjusted based on the validation set to obtain the optimal hyperparameters, and then the optimal detection model for the placement angle of the sole and upper materials is obtained.
[0054] In one embodiment, such as Figure 8 As shown, in step S30, the total loss function is constructed using the Wasserstein distance loss, specifically including: S31: Convert the rotated target detection box into a two-dimensional Gaussian distribution N(m,∑): m = (x, y), where (x, y) represents the coordinates of the center point of the rotated object detection box, w and h represent the width and height of the rotated object detection box, respectively, θ represents the horizontal angle of the long side of the rotated object detection box to the right, R is the rotation matrix, S is the diagonal vector of the eigenvalues, and m is the mean of the Gaussian distribution.
[0055] Specifically, the rotated object detection box is represented as (x, y, w, h, θ), where (x, y) represents the coordinates of the center point of the rotated object detection box, w and h represent the width and height of the rotated object detection box, respectively, and θ represents the horizontal angle to the right of the longer side of the rotated object detection box. The rotated prediction box and the rotated annotation box are converted into two-dimensional Gaussian distributions using a transformation formula, as follows: Figure 3 As shown, Figure 3 To model a two-dimensional Gaussian distribution, the rotating frame B(x,y,w,h,θ) is first transformed into a two-dimensional Gaussian distribution N(m,∑). In the rotating rectangle B(x,y,w,h,θ), x and y are the coordinates of the center point of the rectangle, and w, h, and θ are the width, height, and rotation angle of the rotating rectangle, respectively. After the transformation into a two-dimensional Gaussian distribution N(m,∑), m is determined by the coordinates of the center point of the rotating rectangle, and ∑ is determined by the width, height, and rotation angle of the rotating rectangle.
[0056] S32: Convert the rotated prediction box and rotated label box into their corresponding two-dimensional Gaussian distribution representations, and calculate the Wasserstein distance between the two two-dimensional Gaussian distributions:
[0057] Specifically, after converting the rotated prediction box and the rotated label box into two-dimensional Gaussian distributions, the Wasserstein distance between the two two-dimensional Gaussian distributions is calculated.
[0058] S33: Use a nonlinear transformation f to convert the Wasserstein distance into an affinity metric τ+f(d) to represent the IoU between two bounding boxes. 2 ), where f() represents a nonlinear function, τ represents a hyperparameter, and the formula for calculating the Wasserstein distance loss is: Where d is the square root of the Wasserstein distance between the rotated prediction box and the rotated label box.
[0059] Specifically, since Wasserstein is very sensitive to large errors, a nonlinear transformation f is used to convert the Wasserstein distance into an affinity metric τ+f(d) that is similar to the IoU between two bounding boxes. 2 ), where f() represents a nonlinear function, with respect to the Wasserstein distance d 2 The loss is transformed to make it smoother and more expressive; the hyperparameter τ modulates the entire loss.
[0060] Furthermore, after calculating the Wasserstein distance between the two two-dimensional Gaussian distributions, the Wasserstein distance loss formula is applied: Calculate the Wasserstein distance loss.
[0061] The table below shows ablation experiments on the DOTA dataset with regression loss form and hyperparameters based on the improved YOLOv5 network model, where the base detector is RetinaNet. The data in the table shows that the improved YOLOv5 network model exhibits optimal performance when f() = sqrt and τ = 2.
[0062] That is, f() = sqrt, the optimal hyperparameter τ = 2, and finally the Wasserstein distance loss formula is obtained:
[0063] S34: Label Classification Loss L cls And confidence loss L conf The binary cross-entropy loss function is used: Where N is the total number of label categories, x i y is the predicted value for the current category. i This represents the probability of the current label category after the activation function. It is the actual value of the current tag category.
[0064] In this embodiment, the label classification loss refers to the predicted category of the target. The confidence loss refers to the degree of confidence of the predicted bounding box, with a value ranging from 0 to 1. The higher the value, the more likely the target is to exist in the predicted bounding box.
[0065] Specifically, the similarity between the label categories of the predicted bounding box and the labeled bounding box is calculated using label classification loss, and the presence of target image features in the predicted bounding box is calculated using confidence loss.
[0066] S35: The label classification loss, confidence loss, and Wasserstein distance loss constitute the total loss function. The formula for calculating the total loss function (Loss) is as follows: Loss = L gwd +L cls +L conf , The YOLOv5-GWD model is constructed based on the total loss function Loss.
[0067] Specifically, the total loss function is obtained by weighting the calculation results of the label classification loss, confidence loss, and Wasserstein distance loss, and the accuracy of the detection result is judged based on the total loss function.
[0068] In one embodiment, such as Figure 9 As shown, in step S50, the image of the object to be tested is acquired, and the image is input into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle. Specifically, this includes: S51: Perform type processing on the image to be tested.
[0069] In this embodiment, type processing refers to identifying and converting the data type of the image to be tested.
[0070] Specifically, the image to be detected is type-identified to determine whether it is consistent with the data type used for model training. When the data type of the image to be detected is inconsistent with the data type used for model training, the image to be detected is type-processed. Type processing includes cropping, translation, changing brightness, adding noise, rotation, flipping, and type conversion.
[0071] S52: Generate a detection set based on the type processing results.
[0072] In this embodiment, the detection set refers to the set of images to be detected.
[0073] Specifically, after processing the data type of the image to be tested, i.e., confirming that the data type of the image to be tested is consistent with the data type of the model training, the processed image to be tested is used to construct a detection set.
[0074] S53: Input the detection set into the shoe sole and upper material placement angle detection model to obtain the detection results of the object placement angle.
[0075] Specifically, the detection set is input into the trained optimal model for detecting the placement angle of sole and upper materials to obtain the detection results of the object placement angle. In one embodiment, the detection performance of two detection algorithms, RRPN and R3De, and the improved YOLOv5-GWD on sole and upper materials is compared. RRPN performs poorly in detecting the pose of the four sole and upper materials, with an average accuracy of only 0.849 for soles and three upper materials, a detection speed of 26.3 frames per second, and excessive loss in the angle of the rotating frame, failing to accurately reflect the pose angle of the sole and upper materials. R3Det, compared to... RRPN shows an overall improvement in bounding box quality, with an average accuracy of 0.901 for soles and three upper materials, and a detection speed of 47.6 frames per second. However, it still suffers from significant angle loss and cannot accurately detect the angle information of objects. Compared to the other two detection algorithms, YOLOv5-GWD shows improvements in average accuracy, average precision, and detection speed, with an average precision of 0.957 and a detection speed of 58.8 frames per second. It can accurately detect the angle information of soles and three upper materials, and the quality of its bounding boxes is significantly improved compared to other rotating target detection algorithms.
[0076] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0077] In one embodiment, a device for detecting the placement angle of sole and upper materials is provided, which corresponds one-to-one with the method for detecting the placement angle of sole and upper materials in the above embodiments. For example... Figure 10 As shown, the device for detecting the placement angle of the sole and upper materials includes an image acquisition module, an image feature annotation module, a total loss function construction module, a model training module, and a real-time detection module. Detailed descriptions of each functional module are as follows: The image acquisition module is used to acquire images of different types of shoe sole and upper materials and to perform data augmentation on the images. The image feature annotation module is used to annotate the features of the image using a rotated target detection box based on the data augmentation results, and to obtain a rotated annotation box. The annotated target images are then used to construct a training set and a validation set according to a preset division ratio. The total loss function construction module is used to construct the total loss function using Wasserstein distance loss. The model training module inputs the training set and validation set into the YOLOv5 network model, and improves the YOLOv5 network model according to the total loss function for training, so as to obtain the detection model of the placement angle of the sole and upper materials. The real-time detection module acquires a test image carrying the object to be tested, inputs the test image into the shoe sole and upper material placement angle detection model, and obtains the detection result of the object placement angle.
[0078] Optionally, the image acquisition module includes: The image enhancement processing type acquisition module is used to acquire the image enhancement processing type, which includes cropping, translation, brightness adjustment, noise addition, rotation, and flipping. The data augmentation module is used to randomly select one or more image augmentation processing types and perform data augmentation processing on each image.
[0079] Optionally, the model training module includes: The data receiving module is used to receive the training set and the validation set; The prediction box acquisition module is used to train the training set based on the YOLOv5 network model and obtain the rotated prediction boxes of image features from the training process. The total loss function calculation module is used to calculate the distance index between the rotated prediction box and the rotated label box based on the total loss function; The model validation module is used to select the optimal rotation prediction box based on the distance calculation results and validate the optimal rotation prediction box based on the validation set. The optimal model acquisition module obtains the detection model for the placement angle of the sole and upper materials based on the verification results.
[0080] Optionally, the total loss function building blocks include: The data conversion module is used to convert the rotated target detection box into a two-dimensional Gaussian distribution; The Wasserstein distance calculation module is used to convert the rotated prediction box and rotated label box into their corresponding two-dimensional Gaussian distribution representations and to calculate the Wasserstein distance between the two two-dimensional Gaussian distributions. The Wasserstein distance loss calculation module is used to convert the Wasserstein distance into an affinity metric τ+f(d) representing the IoU between two bounding boxes using a nonlinear transformation f. 2), where f() represents a nonlinear function and τ represents a hyperparameter, to calculate the Wasserstein distance loss; The remaining loss calculation modules include label classification loss L. cls And confidence loss L conf The binary classification cross-entropy loss function is used; the total loss function construction module is used to construct the total loss function from the label classification loss, confidence loss and Wasserstein distance loss, calculate the total loss function Loss, and construct the YOLOv5-GWD model based on the total loss function Loss.
[0081] Optional, the real-time detection module includes: The type processing module is used to perform type processing on the image to be tested; The detection set generation module is used to generate a detection set based on the type processing results; The detection result acquisition module is used to input the detection set into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle.
[0082] Specific limitations regarding the device for detecting the placement angle of sole and upper materials can be found in the limitations of the method for detecting the placement angle of sole and upper materials described above, and will not be repeated here. Each module in the aforementioned device for detecting the placement angle of sole and upper materials can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0083] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores training parameter data and detection results of the optimal sole and upper placement angle detection model. The network interface communicates with external terminals via a network. When executed by the processor, the computer program implements a method for detecting the placement angle of sole and upper materials.
[0084] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Images of shoes with different types of sole and upper materials are collected and then augmented. Based on the data augmentation results, the features of the image are labeled using a rotated target detection box to obtain a rotated labeled box. The labeled target images are then used to construct a training set and a validation set according to a preset division ratio. Construct the total loss function using Wasserstein distance loss; The training and validation sets are input into the YOLOv5 network model, and the YOLOv5 network model is improved according to the total loss function for training, so as to obtain the detection model of the placement angle of the sole and upper materials. Acquire a test image containing the object to be tested, input the test image into the shoe sole and upper material placement angle detection model, and obtain the detection result of the object placement angle.
[0085] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Images of shoes with different types of sole and upper materials are collected and then augmented. Based on the data augmentation results, the features of the image are labeled using a rotated target detection box to obtain a rotated labeled box. The labeled target images are then used to construct a training set and a validation set according to a preset division ratio. Construct the total loss function using Wasserstein distance loss; The training and validation sets are input into the YOLOv5 network model, and the YOLOv5 network model is improved according to the total loss function for training, so as to obtain the detection model of the placement angle of the sole and upper materials. Acquire a test image containing the object to be tested, input the test image into the shoe sole and upper material placement angle detection model, and obtain the detection result of the object placement angle.
[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0088] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for detecting the placement angle of shoe sole and upper materials, characterized in that, The method for detecting the posture of the sole and upper materials includes: Images of shoes with different types of sole and upper materials are collected, and the images are augmented. Based on the data augmentation results, the features of the image are labeled using a rotated target detection box to obtain a rotated labeled box. The labeled target images are then used to construct a training set and a validation set according to a preset division ratio. Construct the total loss function using Wasserstein distance loss; The training set and the validation set are input into the YOLOv5 network model, and the YOLOv5 network model is trained by improving it according to the total loss function to obtain the detection model for the placement angle of the sole and upper materials. A test image containing the object to be tested is acquired, and the test image is input into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle.
2. The method for detecting the placement angle of the sole and upper according to claim 1, characterized in that, The acquisition of images containing different types of shoe sole and upper materials, followed by data augmentation of the images, specifically includes: Obtain the image enhancement processing type, which includes cropping, translation, brightness adjustment, noise addition, rotation, and flipping; One or more of the image enhancement processing types are randomly selected, and data enhancement processing is performed on each of the images.
3. The method for detecting the placement angle of the sole and upper according to claim 1, characterized in that, The process of inputting the training set and the validation set into the YOLOv5 network model, and training the YOLOv5 network model by improving it according to the total loss function, to obtain the shoe sole and upper material placement angle detection model, specifically includes: Receive the training set and the validation set; The training set is trained according to the YOLOv5 network model, and the rotation prediction box of image features is obtained from the training process; The distance index between the rotated prediction box and the rotated label box is calculated based on the total loss function; The optimal rotation prediction box is selected based on the distance calculation results, and the optimal rotation prediction box is verified based on the verification set; the detection model of the placement angle of the sole and upper materials is obtained based on the verification results.
4. The method for detecting the placement angle of the sole and upper according to claim 1 or 3, characterized in that, The construction of the total loss function using Wasserstein distance loss specifically includes: The rotated target detection box is converted into a two-dimensional Gaussian distribution N(m,∑): m = (x, y) Where (x,y) represents the coordinates of the center point of the rotating target detection box, w and h represent the width and height of the rotating target detection box, respectively, θ represents the angle between the long side of the rotating target detection box and the horizontal rightward direction, R is the rotation matrix, S is the diagonal vector of the eigenvalues, and m is the mean of the Gaussian distribution. The rotated prediction box and the rotated label box are converted into their corresponding two-dimensional Gaussian distribution representations, and the Wasserstein distance between the two two-dimensional Gaussian distributions is calculated: The Wasserstein distance is transformed into an affinity metric τ+f(d) to represent the IoU between two bounding boxes using a nonlinear transformation f. 2 ), where f() represents a nonlinear function and τ represents a hyperparameter, the formula for calculating the Wasserstein distance loss is: Where d is the square root of the Wasserstein distance between the rotated prediction box and the rotated label box; Label classification loss L cls And confidence loss L conf The binary cross-entropy loss function is used: Where N is the total number of label categories, x i y is the predicted value for the current category. i This represents the probability of the current label category after the activation function. It is the actual value of the current tag category; The label classification loss, the confidence loss, and the Wasserstein distance loss constitute the total loss function, and the formula for calculating the total loss function (Loss) is as follows: Loss=L gwd +L cls +L conf , The YOLOv5-GWD model is constructed based on the total loss function Loss.
5. The method for detecting the placement angle of the sole and upper according to claim 1, characterized in that, The process of acquiring a test image carrying the object to be tested, and inputting the test image into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle, specifically includes: The image to be tested will be processed according to its type. A detection set is generated based on the processing results of the described type; The detection set is input into the shoe sole and upper material placement angle detection model to obtain the detection result of the object placement angle.
6. A device for detecting the placement angle of a shoe sole and upper, characterized in that, The shoe sole and upper placement angle detection device includes: The image acquisition module is used to acquire images of different types of shoe sole and upper materials, and to perform data augmentation on the images; The image feature annotation module is used to annotate the features of the image using a rotated target detection box based on the data augmentation result, to obtain a rotated annotation box, and to construct a training set and a validation set for the annotated target image according to a preset division ratio; The total loss function construction module is used to construct the total loss function using Wasserstein distance loss. The model training module inputs the training set and the validation set into the YOLOv5 network model, and improves the YOLOv5 network model according to the total loss function to train it, thereby obtaining a detection model for the placement angle of the sole and upper materials. The real-time detection module acquires a test image carrying the object to be tested, inputs the test image into the shoe sole and upper material placement angle detection model, and obtains the detection result of the object placement angle.
7. The shoe sole and upper placement angle detection device according to claim 6, characterized in that, The image acquisition module includes: An image enhancement processing type acquisition module is used to acquire the image enhancement processing type, which includes cropping, translation, brightness adjustment, noise addition, rotation, and flipping. The data augmentation processing module is used to randomly select one or more of the image augmentation processing types and perform data augmentation processing on each of the images.
8. The shoe sole and upper placement angle detection device according to claim 6, characterized in that, The model training module includes: A data receiving module is used to receive the training set and the validation set; The prediction box acquisition module is used to train the training set according to the YOLOv5 network model and acquire the rotation prediction box of image features from the training process. The total loss function calculation module is used to calculate the distance index between the rotated prediction box and the rotated annotation box based on the total loss function; The model validation module is used to select the optimal rotation prediction box based on the distance calculation results and validate the optimal rotation prediction box based on the validation set. The optimal model acquisition module obtains the detection model for the placement angle of the sole and upper materials based on the verification results.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for detecting the placement angle of the sole and upper as described in any one of claims 1 to 5.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for detecting the placement angle of the sole and upper as described in any one of claims 1 to 5.