Safety shoe defect detection method and device, computer equipment and storage medium

Abstract

The invention provides a safety shoe defect detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a first visual angle depth image, a second visual angle depth image and a third visual angle depth image corresponding to a to-be-detected safety shoe; safety shoe contours and depth maximum value points corresponding to the first visual angle depth image, the second visual angle depth image and the third visual angle depth image are obtained respectively, so that a first segmentation point set, a second segmentation point set and a third segmentation point set are generated and used for segmenting the first visual angle depth image, the second visual angle depth image and the third visual angle depth image; obtaining a plurality of first image blocks, second image blocks and third image blocks; respectively acquiring a depth change rate corresponding to each image block; extending the corresponding image blocks according to the depth change rate to obtain a first plane image, a second plane image and a third plane image; and performing template matching on each plane image and a preset template image corresponding to the safety shoes to complete defect detection of the safety shoes.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a safety shoe defect detection method, device, computer equipment and storage medium. Background Art

[0002] In modern industrial production and many specific work scenarios, safety shoes are important personal protective equipment, and their quality and safety are of vital importance. Traditional safety shoe inspection mainly relies on manual visual inspection, which is not only inefficient, but also easily affected by the subjective factors of the inspectors, making it difficult to ensure the accuracy and consistency of the inspection results. With the rapid development of industrial automation and intelligent manufacturing, there is an urgent need for an efficient, accurate and objective automated inspection method for defect detection in the safety shoe production process.

[0003] At present, some existing image detection technologies have been applied in the field of product quality inspection, but there are still many problems in safety shoe defect detection. Ordinary two-dimensional image detection methods are difficult to fully obtain the three-dimensional structural information of safety shoes. For some defects hidden inside the shoe body or difficult to present in two-dimensional images due to light and angle problems, the detection effect is not good. Although some three-dimensional detection technologies have been tried, there are often problems such as incomplete data collection, inaccurate image segmentation, and inefficient feature extraction and matching methods, which cannot meet the actual needs of high-precision and high-speed defect detection on the safety shoe production line. Summary of the invention

[0004] The present application provides a safety shoe defect detection method, device, computer equipment and storage medium, aiming to solve the problems of incomplete data collection, inaccurate image segmentation, and insufficient efficiency of feature extraction and matching methods when the existing three-dimensional detection technology is used for safety shoe detection, and cannot meet the actual needs of high-precision and high-speed defect detection on the safety shoe production line.

[0005] In a first aspect, the present application provides a safety shoe defect detection method, characterized in that the method comprises:

[0006] Acquire a first-perspective depth image, a second-perspective depth image, and a third-perspective depth image corresponding to the safety shoes to be tested; an image acquisition device corresponding to the first-perspective depth image is arranged relative to an image acquisition device corresponding to the second-perspective depth image, and an image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image;

[0007] Respectively obtain the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image;

[0008] Generate a first segmentation point set, a second segmentation point set, and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively;

[0009] Segmenting the first perspective depth image, the second perspective depth image, and the third perspective depth image according to the first segmentation point set, the second segmentation point set, and the third segmentation point set, respectively, to obtain a plurality of first image blocks, second image blocks, and third image blocks;

[0010] Respectively obtaining the depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks;

[0011] Extending each of the first image blocks, each of the second image blocks, and each of the third image blocks corresponding to the depth change rate to obtain a first plane image, a second plane image, and a third plane image;

[0012] Template matching is performed on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection of the safety shoe.

[0013] In some embodiments, the method of respectively obtaining the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image includes: obtaining preset size information corresponding to the safety shoe; the preset size information includes component size information corresponding to a plurality of preset components, and the preset components include at least a sole, an upper, and a toe cap; establishing a mathematical model corresponding to the preset size information; and fitting the mathematical model to the first perspective depth image, the second perspective depth image, and the third perspective depth image based on a random sampling consistency algorithm, respectively, to obtain the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively.

[0014] In some embodiments, before respectively obtaining the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, it also includes: obtaining RGB image information of the safety shoes taken by a preset RGB acquisition device; the RGB image information at least includes color information and texture information; determining the contour information of the safety shoes according to the color information and texture information; obtaining a thermal imaging image corresponding to the safety shoes taken by a preset infrared acquisition device; obtaining temperature change information corresponding to the safety shoes according to the thermal imaging image; and mapping the contour information and the temperature change information to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively.

[0015] In some embodiments, the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively, include: obtaining correlation information and complementarity information between the first perspective depth image, the second perspective depth image, and the third perspective depth image; establishing a multi-perspective joint segmentation model according to the correlation information and complementarity information; respectively inputting the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image into the joint segmentation model, and the joint segmentation model generates the first segmentation point set, the second segmentation point set, and the third segmentation point set.

[0016] In some embodiments, the method of extending each of the first blocks, each of the second blocks, and each of the third blocks according to the depth change rate to obtain a first plane image, a second plane image, and a third plane image includes: dividing each of the first blocks, each of the second blocks, and each of the third blocks according to a plurality of preset areas to determine the preset area corresponding to each block; the preset areas at least include a sole area, an upper area, and an upper area; according to a preset extension strategy and a depth change rate corresponding to each of the preset areas, extending each of the first blocks, each of the second blocks, and each of the third blocks corresponding to the preset areas to obtain a first plane image, a second plane image, and a third plane image; wherein the preset extension strategy corresponding to the sole area is a linear extension; and the preset extension strategies corresponding to the upper area and the upper area are extension methods based on curve fitting or surface reconstruction.

[0017] Exemplarily, extending each first block, each second block and each third block corresponding to the preset area according to the preset extension strategy and depth change rate corresponding to each preset area includes: obtaining structural information corresponding to the safety shoes; obtaining constraint conditions corresponding to each preset area according to the structural information; the constraint conditions at least include the flatness of the sole area, the curvature corresponding to the upper area and the shape corresponding to the upper area; extending each first block, each second block and each third block corresponding to the preset area according to the preset extension strategy, depth change rate and constraint conditions corresponding to each preset area.

[0018] Exemplarily, before extending each of the first tiles, each of the second tiles, and each of the third tiles corresponding to the preset areas according to the preset extension strategy and the depth change rate corresponding to each of the preset areas, it also includes: obtaining material properties corresponding to each of the preset areas; the material properties include at least elasticity or hardness; and correcting the depth change rate corresponding to each of the first tiles, each of the second tiles, and each of the third tiles in each of the preset areas according to the material properties of each of the preset areas.

[0019] In a second aspect, the present application provides a safety shoe defect detection device, the device comprising:

[0020] An image acquisition unit is used to acquire a first-perspective depth image, a second-perspective depth image, and a third-perspective depth image corresponding to the safety shoes to be detected; the image acquisition device corresponding to the first-perspective depth image is arranged relative to the image acquisition device corresponding to the second-perspective depth image, and the image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image;

[0021] An extreme value acquisition unit, used to respectively acquire the safety shoe contour and depth maximum point corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image;

[0022] A point set acquisition unit, configured to generate a first segmentation point set, a second segmentation point set, and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively;

[0023] A block acquisition unit, configured to segment the first perspective depth image, the second perspective depth image, and the third perspective depth image according to the first segmentation point set, the second segmentation point set, and the third segmentation point set, respectively, to acquire a plurality of first blocks, second blocks, and third blocks;

[0024] a change acquisition unit, configured to respectively acquire a depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks;

[0025] A picture block extension unit, configured to extend each of the first picture blocks, each of the second picture blocks, and each of the third picture blocks according to the depth change rate, to obtain a first plane image, a second plane image, and a third plane image;

[0026] The detection completion unit is used to perform template matching on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection on the safety shoe.

[0027] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the safety shoe defect detection method provided in any embodiment of the present application when executing the computer program.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements a safety shoe defect detection method as provided in any embodiment of the present application.

[0029] The present application provides a method, device, computer equipment and storage medium for detecting defects in safety shoes. The method first obtains a first-perspective depth image, a second-perspective depth image and a third-perspective depth image corresponding to the safety shoes to be detected. There are clear requirements for the setting of the image acquisition device. The image acquisition device corresponding to the first-perspective depth image is relatively set to the image acquisition device corresponding to the second-perspective depth image, and the image acquisition device corresponding to the third-perspective depth image is vertically set to the image acquisition device corresponding to the first-perspective depth image. Through multiple image acquisition devices arranged at specific angles, data can be collected on safety shoes from different directions to provide basic data for subsequent comprehensive inspections.

[0030] For the acquired first-perspective depth image, second-perspective depth image, and third-perspective depth image, the corresponding safety shoe contour and depth maximum point are obtained respectively. The safety shoe contour can outline the approximate shape of the shoe at that perspective, and the depth maximum point reflects the prominent feature position of the shoe surface in terms of depth, which are important bases for subsequent operations such as image segmentation.

[0031] According to the safety shoe contours and depth maximum points corresponding to the images of each perspective obtained above, the first segmentation point set, the second segmentation point set and the third segmentation point set are generated respectively. These segmentation point sets are actually the key reference data sets for the subsequent image segmentation operation, and the points contained in them determine the specific location and range of the segmentation.

[0032] The first perspective depth image, the second perspective depth image and the third perspective depth image are segmented respectively using the generated first segmentation point set, the second segmentation point set and the third segmentation point set, thereby obtaining a plurality of first image blocks, a second image block and a third image block. Dividing the originally complete depth images of each perspective into a plurality of small image blocks helps to analyze and process the features of different parts of the image in more detail.

[0033] Next, the depth change rate corresponding to each first block, each second block, and each third block is obtained respectively. The depth change rate can reflect the characteristics of the speed of the depth change inside the block, and then the corresponding blocks are extended according to the depth change rate, and finally the first plane image, the second plane image, and the third plane image are obtained. In this way, the original depth image block is converted into a plane image, which is more convenient for subsequent matching and other operations, while also retaining the characteristic information of each part of the shoe.

[0034] Finally, the obtained first plane image, second plane image and third plane image are matched with the preset template image corresponding to the safety shoe. By comparing the difference between the actual image and the preset standard template image, it is judged whether the safety shoe has defects. If there is a difference beyond a certain range, it means that there may be a defect here, thereby completing the defect detection of the safety shoe.

[0035] The provided method has at least the following beneficial effects:

[0036] 1. More comprehensive data collection: The depth image collection method with three different perspectives (first perspective, second perspective and third perspective perpendicular to the first perspective) can obtain the appearance and depth feature data of safety shoes in different directions in an all-round way compared to the collection from a single perspective or partial perspective, avoiding the omission of some defect features due to limited data collection angles, so that subsequent inspections have a more complete data basis, which is more in line with the actual needs of the safety shoe production line for comprehensive inspection.

[0037] 2. More accurate image segmentation: By first obtaining the contour and depth maximum points of the safety shoes to generate a segmentation point set, and then performing image segmentation based on the segmentation point set, compared to some conventional image segmentation methods that lack precise basis, this segmentation strategy based on the characteristics of the safety shoes themselves can divide the image more accurately and accurately divide the different parts of the safety shoes into different blocks, which is helpful for subsequent more detailed analysis of the characteristics of each part and improves the accuracy of the overall detection.

[0038] 3. Feature extraction and matching are more efficient: The method of first extending the block depth change rate to obtain a plane image, and then matching it with the preset template image. On the one hand, the extraction and utilization of the depth change rate feature can better reflect the characteristics of the block itself, which helps to retain key features during the extension process; on the other hand, matching with the preset template image and comparing with a standard reference makes defect detection more targeted and efficient, which can meet the requirements of high-precision and high-speed defect detection on the safety shoe production line, help improve production efficiency, ensure product quality, and promptly discover defective safety shoe products to prevent defective products from entering the market.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 is a schematic flow chart of the steps of a safety shoe defect detection method provided by an embodiment of the present application;

[0042] Figure 2 is a schematic block diagram of the structure of a safety shoe defect detection device provided by an embodiment of the present application;

[0043] Figure 3 It is a schematic block diagram of the structure of a computer device provided in one embodiment of the present application.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0047] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0048] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0049] It should also be understood that the term “and / or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0050] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0051] In modern industrial production and many specific work scenarios, safety shoes are important personal protective equipment, and their quality and safety are of vital importance. Traditional safety shoe inspection mainly relies on manual visual inspection, which is not only inefficient, but also easily affected by the subjective factors of the inspectors, making it difficult to ensure the accuracy and consistency of the inspection results. With the rapid development of industrial automation and intelligent manufacturing, there is an urgent need for an efficient, accurate and objective automated inspection method for defect detection in the safety shoe production process.

[0052] At present, some existing image detection technologies have been applied in the field of product quality inspection, but there are still many problems in safety shoe defect detection. Ordinary two-dimensional image detection methods are difficult to fully obtain the three-dimensional structural information of safety shoes. For some defects hidden inside the shoe body or difficult to present in two-dimensional images due to light and angle problems, the detection effect is not good. Although some three-dimensional detection technologies have been tried, there are often problems such as incomplete data collection, inaccurate image segmentation, and inefficient feature extraction and matching methods, which cannot meet the actual needs of high-precision and high-speed defect detection on the safety shoe production line.

[0053] To solve the above problems, the present application provides a safety shoe defect detection method, which can be carried out on a computer device, and the computer device can be deployed on a single server or a server cluster. It can also be deployed on a handheld terminal, a laptop computer, a wearable device or a robot, etc.

[0054] like Figure 1 As shown, the provided safety shoe defect detection method includes steps S101 to S107. The details are as follows:

[0055] Step S101. Obtain a first-perspective depth image, a second-perspective depth image and a third-perspective depth image corresponding to the safety shoes to be inspected; an image acquisition device corresponding to the first-perspective depth image is arranged relative to the image acquisition device corresponding to the second-perspective depth image, and an image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image.

[0056] Specifically, the purpose of step S101 is to obtain depth images of safety shoes from different perspectives. The depth image provides depth information of the surface of the object, which can help the detection method to more accurately identify the shape, structure and surface features of the safety shoes. Specifically, the image acquisition devices of the first perspective and the second perspective are relatively set, which means that these two perspectives can capture both sides of the safety shoes, and the third perspective is perpendicular to the first perspective and can capture the top or bottom of the safety shoes. This multi-perspective setting can ensure that the information of the safety shoes is comprehensively obtained from multiple angles.

[0057] Suppose there is a safety shoe detection scenario, and three depth cameras are used to shoot the safety shoes from the front, back, and top. The front camera captures the depth image of the front, the rear camera captures the depth image of the back, and the top camera captures the depth image of the top.

[0058] Multi-perspective depth images can supplement details that cannot be captured by a single perspective, thereby improving the comprehensiveness and accuracy of defect detection.

[0059] Step S102: respectively obtain the safety shoe contours and depth maximum points corresponding to the first-view depth image, the second-view depth image, and the third-view depth image.

[0060] Specifically, after obtaining the depth image, it is necessary to further extract the contour and depth maximum points of the safety shoes. The contour information can help determine the boundary of the safety shoes, while the depth maximum points can highlight the surface features and structure of the safety shoes.

[0061] The contour of the safety shoes is extracted by using an edge detection algorithm (such as Canny edge detection). For the extraction of the depth maximum point, a local maximum algorithm can be used to find the maximum depth value point in each local area in the depth image.

[0062] The extraction of contour and depth maximum points helps with image segmentation and feature extraction in subsequent steps, thereby improving the accuracy and efficiency of detection.

[0063] Step S103: Generate a first segmentation point set, a second segmentation point set and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image and the third perspective depth image respectively.

[0064] Specifically, the extracted contours and depth maximum points are used as segmentation points to generate a segmentation point set, which is used for subsequent image segmentation to ensure that each image block contains important surface features and structural information.

[0065] Assume that the safety shoe contour extracted in the first-view depth image consists of 50 edge points and the depth maximum point consists of 20 local maximum depth value points. Combine these points to generate a first segmentation point set containing 70 segmentation points. The same method is applied to the second-view and third-view depth images.

[0066] The generated segmentation point set can ensure the accuracy of image segmentation, avoid the loss of important features, and improve the efficiency and reliability of subsequent processing.

[0067] Step S104: segment the first perspective depth image, the second perspective depth image and the third perspective depth image according to the first segmentation point set, the second segmentation point set and the third segmentation point set respectively to obtain a plurality of first image blocks, second image blocks and third image blocks.

[0068] Specifically, the depth image is segmented using the generated segmentation point set to generate multiple blocks. Each block contains a portion of the surface information of the safety shoe, which is convenient for subsequent depth change rate calculation and defect detection.

[0069] The depth image is segmented according to the segmentation point set by using a segmentation algorithm (such as the Watershed algorithm). Assuming that each segmentation point set generates 10 blocks, then each view will generate 10 depth image blocks.

[0070] Image segmentation can decompose a complex image into multiple simple small blocks, which is convenient for detailed analysis block by block, improving processing speed and defect detection accuracy.

[0071] Step S105: respectively obtain the depth change rate corresponding to each first image block, each second image block and each third image block.

[0072] Specifically, the depth change rate inside each block is calculated. The depth change rate reflects the irregularity of the surface inside the image block and helps to identify potential defect areas.

[0073] For example, for each tile, the depth value changes of all pixels inside it are calculated, and the depth change rate is quantified using the standard deviation or gradient method. For example, an area with a large standard deviation may indicate that the surface has protrusions or depressions.

[0074] The calculation of the depth change rate can quantify the surface features of the image block, which helps to identify subtle defects and improve the sensitivity of detection.

[0075] Step S106: Extend each corresponding first image block, each second image block and each third image block according to the depth change rate to obtain a first plane image, a second plane image and a third plane image.

[0076] Specifically, each block is extended according to its depth change rate to generate a plane image. The purpose of the extension is to convert the three-dimensional structure information into a two-dimensional plane image to facilitate subsequent template matching.

[0077] Assuming that the depth change rate inside a certain tile is large, it means that the surface of the area has large fluctuations. Interpolation methods (such as bilinear interpolation) can be used to extend the three-dimensional surface of the tile into a flat image.

[0078] The extended plane image retains the three-dimensional structural information of the original image block, while simplifying the complexity of image processing, making it easier to perform template matching and defect detection.

[0079] Step S107: template matching is performed on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection of the safety shoe.

[0080] Specifically, the generated plane image is matched with a preset template image, and the area that does not match the template image is identified as the defective area. The template image can be a standard image of a safety shoe without defects.

[0081] For example, for each plane image, a template matching algorithm (such as normalized cross-correlation matching) is used to compare it with a preset template image. The algorithm calculates the similarity between the two images and marks the areas with large differences.

[0082] Template matching can quickly and accurately identify defective areas, avoiding the subjectivity and inconsistency of manual inspection and improving the accuracy and efficiency of detection.

[0083] The acquisition and processing of multi-view depth images ensures that all parts of the safety shoes can be detected, avoiding the blind spots of single-view detection. Through the extraction of contour and depth maximum points, image segmentation and calculation of depth change rate, potential defects can be identified more accurately. The automated detection method greatly improves the detection speed and reduces the cost and time of manual inspection. The detection method based on image processing reduces the interference of human factors and improves the consistency of detection results. This method can be deployed on a variety of devices to adapt to different detection environments and needs.

[0084] In step S107, the template image should be a standard image of a defect-free safety shoe. These images are obtained through high-precision acquisition equipment (such as high-resolution cameras and 3D scanners), and are processed and annotated to ensure that there are no defects. In order to improve the accuracy of detection, a template image library containing a variety of standard images can be prepared. These images can cover different styles of safety shoes, samples of different production batches, and even images under different production environments.

[0085] For example, a 3D scanner is used to scan defect-free safety shoes to generate a high-precision depth image. The generated depth image is processed to extract contours, generate segmentation point sets, segment the image, calculate the depth change rate and extension, and finally generate a standard plane image. These standard plane images are stored in the template image library for easy reference during the inspection process.

[0086] The key is to choose the right template matching algorithm. Common template matching algorithms include normalized cross correlation matching (NCC), square difference matching (SSD), phase correlation matching (PC), etc. According to the characteristics of safety shoes and detection requirements, choose the most suitable algorithm. For example, normalized cross correlation matching is suitable for situations where the image brightness and contrast change greatly, while square difference matching is suitable for situations where the image brightness and contrast change little.

[0087] For example, the Normalized Cross Correlation (NCC) algorithm is selected because it has good robustness to changes in image brightness and contrast and is suitable for various environments in industrial production. The Phase Correlation (PC) algorithm is used for detecting images with large rotation or translation changes.

[0088] Each generated plane image is matched with a standard plane image in the template image library and the similarity is calculated. Based on the selected template matching algorithm, the similarity between the generated plane image and each template image is calculated. The similarity can be a numerical value indicating the degree of match between the two images. A similarity threshold is set. When the similarity between the generated plane image and the template image is lower than the threshold, the block is considered to have a defect.

[0089] For the first to third plane images, the normalized cross correlation matching (NCC) algorithm is used to calculate their similarity with all standard plane images in the template image library. If the similarity threshold is set to 0.85, when the similarity of a block is lower than 0.85, the block is considered to have defects.

[0090] Mark the tiles with similarity below the threshold as defect areas. Different colors or marker symbols can be used to distinguish different types of defects. According to the characteristics of the defect area (such as shape, size, depth change rate, etc.), it is classified into different defect types, such as cracks, depressions, protrusions, etc.

[0091] For example, if the image blocks with similarity lower than 0.85 are marked in red, it indicates that there may be defects. Further analysis of the characteristics of the red marked area shows that if the depth change rate of the area is large and the shape is irregular, it is classified as a crack; if the depth change rate is positive and the shape is relatively regular, it is classified as a protrusion.

[0092] Summarize the inspection results of all viewing angles and generate a final defect inspection report. The report should include detailed information such as the location, type, size, and depth change rate of each defect area. Visualize the inspection results for easy manual review and confirmation.

[0093] The inspection report lists all areas marked as defects, and provides the location (e.g., toe, heel, etc.) and type (crack, dent, etc.) of each area. In the visual interface, the defective areas are displayed using markers of different colors, and the type and size of each area are marked.

[0094] The accuracy of the test is ensured by providing the test results to the quality inspectors for review. The quality inspectors can feed back the review results to the system, and the system continuously optimizes the template matching algorithm and threshold settings based on the feedback to improve the reliability and accuracy of the test.

[0095] If the test results are submitted to the quality inspector, the quality inspector will review them through the visual interface. If the quality inspector finds that some areas are misjudged as defects, this information can be fed back to the system, and the system will adjust the similarity threshold or optimize the matching algorithm to reduce misjudgments.

[0096] Through template matching, different types of defects of safety shoes can be quickly and accurately identified, improving the accuracy of detection. The template matching algorithm reduces the interference of human factors and ensures the consistency of the test results. The visual display of the test results facilitates the rapid review of quality inspectors and improves work efficiency. Through manual review and feedback mechanisms, the system can be continuously optimized to adapt to different production environments and defect types.

[0097] In some embodiments, the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image are obtained respectively, including: obtaining preset size information corresponding to the safety shoe; the preset size information includes component size information corresponding to multiple preset components, and the preset components include at least a sole, an upper, and a toe cap; establishing a mathematical model corresponding to the preset size information; fitting the mathematical model to the first perspective depth image, the second perspective depth image, and the third perspective depth image based on a random sampling consistency algorithm, respectively, and obtaining the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image respectively.

[0098] The preset size information includes the size information of multiple preset components (such as the sole, the upper and the toe). This information can be obtained by measuring the standard safety shoes and stored in the system. The component size information includes, for example, the length, width and thickness of the sole, the height and width of the upper, the length and width of the toe, etc.

[0099] A mathematical model is established based on the preset size information, and the model can describe the shape and size characteristics of each component of the safety shoe. The parameters of the mathematical model can be the geometric shape (such as a cuboid, a cylinder, etc.) and the specific size value of the component.

[0100] The random sampling consensus algorithm (RANSAC) is an algorithm used to estimate the parameters of a data model, especially in the presence of a large amount of noise and outliers. In this embodiment, RANSAC is used to fit the mathematical model to the depth image and extract the contour and depth maximum points of the safety shoes. RANSAC randomly selects several points in the depth image, builds a preliminary model, and then calculates the fitting error between the remaining points and the model, selects the point set with the smallest error as the inliers, and rebuilds the model until the error of the model is minimized.

[0101] By combining the preset size information with the RANSAC algorithm, the contour and depth maximum points of the safety shoes can be extracted more accurately, reducing the influence of noise and outliers. The automated method reduces the errors of manual operation and improves the reliability of the detection results. The preset size information can cover different styles of safety shoes, making the detection method more universal and flexible.

[0102] In some embodiments, before respectively obtaining the safety shoe contours and depth maximum points corresponding to the first-perspective depth image, the second-perspective depth image, and the third-perspective depth image, it also includes: obtaining RGB image information of the safety shoes taken by a preset RGB acquisition device; the RGB image information includes at least color information and texture information; determining the contour information of the safety shoes based on the color information and texture information; obtaining a thermal imaging image corresponding to the safety shoes taken by a preset infrared acquisition device; obtaining temperature change information corresponding to the safety shoes based on the thermal imaging image; and mapping the contour information and temperature change information to the first-perspective depth image, the second-perspective depth image, and the third-perspective depth image, respectively.

[0103] By using a preset RGB acquisition device (such as an ordinary camera) to capture the RGB image of the safety shoe, the color information and texture information of the safety shoe are obtained. The color information is used to reflect the color distribution on the surface of the safety shoe, which helps to identify areas with abnormal colors. The texture information is used to reflect the texture characteristics of the surface of the safety shoe, which helps to identify areas with abnormal textures. According to the color information and texture information of the RGB image, the contour information of the safety shoe is determined using an edge detection algorithm (such as Canny edge detection) or a contour extraction algorithm (such as morphological operation). Use a preset infrared camera to capture a thermal imaging image of the safety shoe to obtain the temperature distribution on the surface of the safety shoe.

[0104] By reflecting the temperature changes on the surface of safety shoes, it helps to identify abnormal temperature areas caused by material defects or improper temperature control during the production process. The contour information and temperature change information extracted from the RGB image and thermal imaging image are mapped to the depth images of the first, second and third perspectives to assist the defect detection of the depth image.

[0105] Combining the information of RGB images and thermal images can more comprehensively evaluate the surface characteristics of safety shoes and improve the accuracy of defect detection. Changes in color and temperature can identify some defects that are difficult to find in depth images, such as uneven materials and internal structural problems. Auxiliary information can reduce misjudgments, improve detection efficiency, and reduce unnecessary re-inspection work.

[0106] In some embodiments, based on the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively, the method includes: obtaining the correlation information and complementarity information between the first perspective depth image, the second perspective depth image, and the third perspective depth image; establishing a multi-perspective joint segmentation model based on the correlation information and the complementarity information; inputting the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image into the joint segmentation model, and the joint segmentation model generates a first segmentation point set, a second segmentation point set, and a third segmentation point set.

[0107] Analyze the correlation information between the first-perspective, second-perspective, and third-perspective depth images, for example, the relationship between the depth change rates of the same area at different perspectives. Complementarity information is used to analyze the complementarity between depth images from different perspectives, for example, details that cannot be captured from one perspective can be clearly presented from another perspective.

[0108] A multi-view joint segmentation model is established based on correlation and complementarity information. The model can integrate information from multiple perspectives to generate a more accurate set of segmentation points. The joint segmentation model can be trained through machine learning methods (such as neural networks) to improve its segmentation accuracy and robustness.

[0109] The safety shoe contours and depth maximum points corresponding to the first, second, and third perspective depth images are input into the joint segmentation model. The joint segmentation model generates the first segmentation point set, the second segmentation point set, and the third segmentation point set, which are used for subsequent image segmentation and defect detection.

[0110] The multi-view joint segmentation model can comprehensively utilize information from different viewpoints to generate a more accurate set of segmentation points and reduce the error of single view segmentation. The joint segmentation model has better robustness to noise and outliers and can more stably process images in complex production environments. The joint segmentation model can continuously adapt to new defect types and production environments through training, improving the flexibility and reliability of detection.

[0111] Exemplarily, a multi-view joint segmentation model is established based on correlation information and complementary information, including:

[0112] Use a high-precision depth camera to shoot safety shoes from different angles to obtain first-perspective depth images, second-perspective depth images, and third-perspective depth images. Make sure that the depth image of each perspective covers different areas of the safety shoes to obtain comprehensive depth information.

[0113] Use professional image annotation tools (such as LabelMe, VGG Image Annotator, etc.) to annotate the depth image. Annotate the outline and defect areas of the safety shoes and generate standard segmentation labels.

[0114] The depth change rate of each tile is calculated to reflect the concave and convex features of the surface. The surface contour of each perspective depth image is extracted using an edge detection algorithm (such as Canny edge detection).

[0115] Extract the color information of the RGB image corresponding to each viewing angle, such as the pixel value of the RGB channel. Use texture feature extraction methods (such as local binary pattern LBP, gray level co-occurrence matrix GLCM) to extract texture information. Calculate the temperature change rate of each block to reflect the temperature distribution in the thermal imaging image. Mark the temperature abnormal area to assist defect detection.

[0116] By calculating the correlation between depth images at different perspectives, we can understand the similarities and differences of images at different perspectives. We can use methods such as mutual information and correlation coefficient to calculate the correlation.

[0117] Analyze the complementarity between depth images at different perspectives and identify information that cannot be effectively captured at a certain perspective. Methods such as feature complementarity or information gain can be used for analysis.

[0118] Use registration techniques (such as ICP algorithm, feature matching algorithm, etc.) to align depth images from different perspectives to the same coordinate system. Ensure the consistency of images from different perspectives in spatial position for subsequent feature fusion and model training.

[0119] Common deep learning models such as U-Net, SegNet, DeepLab, etc., these models perform well in image segmentation tasks. You can choose the appropriate model architecture according to actual needs. For example, the jump connection of U-Net can better retain high-resolution feature information.

[0120] Organize the aligned multi-view depth images and their related feature information (such as RGB images, thermal images) into a format suitable for model input. Images from different perspectives can be concatenated into a multi-channel input, or images from each perspective can be input into different branches of the model.

[0121] In the early stages of the model (such as the input layer), the feature information of multiple perspectives is fused. This can be achieved through methods such as splicing feature maps and weighted averaging. In the middle and late stages of the model (such as the feature extraction layer and segmentation layer), the feature information of multiple perspectives is fused. This can be achieved through methods such as attention mechanism and feature splicing.

[0122] Prepare a dataset containing multi-view depth images, RGB images, thermal images and their corresponding annotated data. Enhance the training data through rotation, scaling, flipping and other techniques to improve the generalization ability of the model. Common loss functions include cross entropy loss, Dice loss, Jaccard loss, etc. If depth images, RGB images and thermal images are used at the same time, a multi-task loss function can be designed to comprehensively consider the losses of different modalities.

[0123] Set appropriate batch size, learning rate, number of training epochs, etc. Use optimization algorithms (such as Adam, SGD) to train the model and minimize the loss function through gradient descent. Use early stopping to prevent overfitting. Use learning rate decay techniques (such as ReduceLROnPlateau) to dynamically adjust the learning rate.

[0124] Evaluation metrics include:

[0125] Precision: The ratio of defect areas correctly identified by the calculation model to all identified areas.

[0126] Recall: The ratio of defect areas correctly identified by the calculation model to all actual defect areas.

[0127] F1 score: The F1 score is calculated by taking into account the precision and recall.

[0128] Confusion Matrix: Analyzes the effects of different types of defect detection and identifies the misjudgments of the model.

[0129] Adjust the model's hyperparameters, such as learning rate, batch size, number of network layers, etc. Based on the evaluation results, optimize the network structure, such as adding or reducing convolutional layers, using attention mechanisms, etc. Improve the generalization ability of the model through more data enhancement techniques. Use regularization techniques (such as L1 regularization, L2 regularization) to prevent overfitting.

[0130] After the model training is completed, the first-perspective depth image, the second-perspective depth image, and the third-perspective depth image are aligned to the same coordinate system. The depth features, color features, and thermal imaging features of each perspective are extracted. The aligned multi-perspective images and their feature information are input into the trained joint segmentation model. The model generates the first segmentation point set, the second segmentation point set, and the third segmentation point set to represent the outline of the safety shoes and potential defect areas. The segmentation point sets of different perspectives are aggregated to generate a comprehensive segmentation result. The generated segmentation result is compared with the preset standard template image to identify the defect area. Based on the identified defect area, a final defect detection report is generated.

[0131] The above example uses a multi-view joint segmentation model to comprehensively utilize the depth image information from different perspectives to generate a more accurate set of segmentation points and reduce the error of single-view segmentation. The model fuses the feature information from different perspectives, improves the robustness to noise and outliers, and makes the detection results more stable. The automated method reduces the time of manual operation and improves the efficiency of detection. The fusion of multimodal information (such as RGB images and thermal imaging images) enables the model to more comprehensively evaluate the surface features of safety shoes and reduce misjudgments. The model can continuously adapt to new defect types and production environments through training, and has high flexibility and reliability.

[0132] In some embodiments, each corresponding first tile, each second tile and each third tile are extended according to the depth change rate to obtain a first plane image, a second plane image and a third plane image, including: dividing each first tile, each second tile and each third tile according to multiple preset areas to determine the preset area corresponding to each tile; the preset area at least includes a sole area, an upper area and an upper area; according to a preset extension strategy and a depth change rate corresponding to each preset area, each first tile, each second tile and each third tile corresponding to the preset area are extended to obtain a first plane image, a second plane image and a third plane image; wherein the preset extension strategy corresponding to the sole area is linear extension; and the preset extension strategy corresponding to the upper area and the upper area is an extension method based on curve fitting or surface reconstruction.

[0133] Each block (first block, second block and third block) is divided into a plurality of preset regions. The preset regions include at least a sole region, an upper region and a vamp region. The preset region corresponding to each block is determined by analyzing the depth image and a preset region division standard.

[0134] A linear extension strategy is used. The linear extension strategy is suitable for flat areas and can extend the local plane in the depth image into a complete plane image. An extension method based on curve fitting or surface reconstruction is used. Curve fitting is suitable for areas with regular edges, while surface reconstruction is suitable for areas with complex surfaces. For the sole area, linear extension is performed according to the depth change rate to generate a plane image. For the upper area and the vamp area, curve fitting or surface reconstruction is performed according to the depth change rate to generate a smooth plane image.

[0135] Using different extension strategies for different areas can better preserve the characteristics of each area and improve the accuracy and effect of extension. Linear extension is suitable for flat sole areas, which can reduce misjudgment caused by unclear depth changes; curve fitting and surface reconstruction are suitable for complex upper and vamp areas, which can more accurately capture surface features. Targeted extension strategies can reduce unnecessary calculations and improve the efficiency of image processing.

[0136] Exemplarily, according to the preset extension strategy and depth change rate corresponding to each preset area, each first block, each second block and each third block corresponding to the preset area are extended, including: obtaining structural information corresponding to the safety shoes; obtaining constraint conditions corresponding to each preset area according to the structural information; the constraint conditions include at least the flatness of the sole area, the curvature corresponding to the upper area and the shape corresponding to the upper area; according to the preset extension strategy, depth change rate and constraint conditions corresponding to each preset area, each first block, each second block and each third block corresponding to the preset area are extended.

[0137] By obtaining the structural information of safety shoes, including the specific size, shape and curvature of each area. This information can be obtained through high-precision 3D scanning and measurement. The sole should be relatively flat, and the flatness constraints can be set to ensure that the extended plane image conforms to the shape characteristics of the sole. The upper may have a certain curvature, and the curvature constraints can be set to ensure that the extended plane image retains the surface characteristics of the upper. The upper may have different shapes, and the shape constraints can be set to ensure that the extended plane image conforms to the structural characteristics of the upper. Linear extension is performed according to the depth change rate and flatness constraints. Curve fitting or surface reconstruction is performed according to the depth change rate and curvature or shape constraints.

[0138] By introducing constraints, the image can be extended more accurately and the misjudgment caused by inaccurate depth information can be reduced. The constraints ensure that the extended plane image conforms to the actual structural characteristics of the safety shoes and improves the reliability of the detection. The constraints can reduce the impact of noise and improve the robustness of the extension process.

[0139] Exemplarily, before extending each first tile, each second tile, and each third tile corresponding to the preset area according to the preset extension strategy and depth change rate corresponding to each preset area, it also includes: obtaining material properties corresponding to each preset area; the material properties include at least elasticity or hardness; and correcting the depth change rate corresponding to each first tile, each second tile, and each third tile in each preset area according to the material properties of each preset area.

[0140] Obtain the material properties of each preset area (sole, upper and upper), including elasticity, hardness, etc. This information can be obtained through material testing and stored in the system. For areas with high material elasticity (such as the upper), since the deformation of the material may affect the calculation result of the depth change rate, an elasticity correction factor can be added to adjust the depth change rate. For areas with high material hardness (such as the sole), since the material hardly deforms, the depth change rate correction can be reduced or not performed. Extend according to the corrected depth change rate to generate the first plane image, the second plane image and the third plane image.

[0141] Correction based on material characteristics can reduce the depth change rate error caused by material deformation and improve the accuracy of detection. The correction method enables the detection method to adapt to safety shoes of different materials and improves universality. The corrected depth change rate makes the extension process more accurate and reduces misjudgment caused by material characteristics.

[0142] See also Figure 2 , Figure 21 is a schematic diagram of the structure of a safety shoe defect detection device 200 provided in an embodiment of the present application. The safety shoe defect detection device 200 is used to perform the steps of the safety shoe defect detection method shown in any embodiment of the present application. The safety shoe defect detection device 200 can be a single server or a server cluster, or the safety shoe defect detection device 200 can be a terminal, which can be a handheld terminal, a laptop computer, a wearable device or a robot, etc.

[0143] like Figure 2 As shown, the safety shoe defect detection device 200 includes:

[0144] The image acquisition unit 201 is used to acquire a first-perspective depth image, a second-perspective depth image and a third-perspective depth image corresponding to the safety shoes to be detected; the image acquisition device corresponding to the first-perspective depth image is arranged relative to the image acquisition device corresponding to the second-perspective depth image, and the image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image;

[0145] An extreme value acquisition unit 202 is used to respectively acquire the safety shoe contour and depth maximum point corresponding to the first perspective depth image, the second perspective depth image and the third perspective depth image;

[0146] A point set acquisition unit 203 is used to generate a first segmentation point set, a second segmentation point set and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image and the third perspective depth image, respectively;

[0147] A block acquisition unit 204 is used to segment the first perspective depth image, the second perspective depth image and the third perspective depth image according to the first segmentation point set, the second segmentation point set and the third segmentation point set, respectively, to acquire a plurality of first blocks, second blocks and third blocks;

[0148] A change acquisition unit 205 is used to respectively acquire a depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks;

[0149] A block extension unit 206, configured to extend each of the first blocks, each of the second blocks, and each of the third blocks according to the depth change rate to obtain a first plane image, a second plane image, and a third plane image;

[0150] The detection completion unit 207 is used to perform template matching on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection on the safety shoe.

[0151] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the safety shoe defect detection device 200 and each module described above can refer to the corresponding process in the safety shoe defect detection method embodiment described in any embodiment of the present application, and will not be repeated here.

[0152] Figure 1 The corresponding safety shoe defect detection method can be implemented in the form of a computer program, which can be used in Figure 2 Run on the device shown.

[0153] See also Figure 3 , Figure 3 1 is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes a processor, a memory and a network interface connected via a device bus, wherein the memory may include a storage medium and an internal memory.

[0154] The storage medium can store operating devices and computer programs. The computer program includes program instructions, which, when executed, can cause the processor to execute Figure 2 Corresponding to any safety shoe defect detection method.

[0155] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0156] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any safety shoe defect detection method.

[0157] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0159] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0160] Acquire a first-perspective depth image, a second-perspective depth image, and a third-perspective depth image corresponding to the safety shoes to be tested; an image acquisition device corresponding to the first-perspective depth image is arranged relative to an image acquisition device corresponding to the second-perspective depth image, and an image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image;

[0161] Respectively obtain the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image;

[0162] Generate a first segmentation point set, a second segmentation point set, and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively;

[0163] Segmenting the first perspective depth image, the second perspective depth image, and the third perspective depth image according to the first segmentation point set, the second segmentation point set, and the third segmentation point set, respectively, to obtain a plurality of first image blocks, second image blocks, and third image blocks;

[0164] Respectively obtaining the depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks;

[0165] Extending each of the first image blocks, each of the second image blocks, and each of the third image blocks corresponding to the depth change rate to obtain a first plane image, a second plane image, and a third plane image;

[0166] Template matching is performed on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection of the safety shoe.

[0167] In some embodiments, the method of respectively obtaining the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image includes: obtaining preset size information corresponding to the safety shoe; the preset size information includes component size information corresponding to a plurality of preset components, and the preset components include at least a sole, an upper, and a toe cap; establishing a mathematical model corresponding to the preset size information; and fitting the mathematical model to the first perspective depth image, the second perspective depth image, and the third perspective depth image based on a random sampling consistency algorithm, respectively, to obtain the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively.

[0168] In some embodiments, before respectively obtaining the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, it also includes: obtaining RGB image information of the safety shoes taken by a preset RGB acquisition device; the RGB image information at least includes color information and texture information; determining the contour information of the safety shoes according to the color information and texture information; obtaining a thermal imaging image corresponding to the safety shoes taken by a preset infrared acquisition device; obtaining temperature change information corresponding to the safety shoes according to the thermal imaging image; and mapping the contour information and the temperature change information to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively.

[0169] In some embodiments, the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively, include: obtaining correlation information and complementarity information between the first perspective depth image, the second perspective depth image, and the third perspective depth image; establishing a multi-perspective joint segmentation model according to the correlation information and complementarity information; respectively inputting the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image into the joint segmentation model, and the joint segmentation model generates the first segmentation point set, the second segmentation point set, and the third segmentation point set.

[0170] In some embodiments, the method of extending each of the first blocks, each of the second blocks, and each of the third blocks according to the depth change rate to obtain a first plane image, a second plane image, and a third plane image includes: dividing each of the first blocks, each of the second blocks, and each of the third blocks according to a plurality of preset areas to determine the preset area corresponding to each block; the preset areas at least include a sole area, an upper area, and an upper area; according to a preset extension strategy and a depth change rate corresponding to each of the preset areas, extending each of the first blocks, each of the second blocks, and each of the third blocks corresponding to the preset areas to obtain a first plane image, a second plane image, and a third plane image; wherein the preset extension strategy corresponding to the sole area is a linear extension; and the preset extension strategies corresponding to the upper area and the upper area are extension methods based on curve fitting or surface reconstruction.

[0171] Exemplarily, extending each first block, each second block and each third block corresponding to the preset area according to the preset extension strategy and depth change rate corresponding to each preset area includes: obtaining structural information corresponding to the safety shoes; obtaining constraint conditions corresponding to each preset area according to the structural information; the constraint conditions at least include the flatness of the sole area, the curvature corresponding to the upper area and the shape corresponding to the upper area; extending each first block, each second block and each third block corresponding to the preset area according to the preset extension strategy, depth change rate and constraint conditions corresponding to each preset area.

[0172] Exemplarily, before extending each of the first tiles, each of the second tiles, and each of the third tiles corresponding to the preset areas according to the preset extension strategy and the depth change rate corresponding to each of the preset areas, it also includes: obtaining material properties corresponding to each of the preset areas; the material properties include at least elasticity or hardness; and correcting the depth change rate corresponding to each of the first tiles, each of the second tiles, and each of the third tiles in each of the preset areas according to the material properties of each of the preset areas.

[0173] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the safety shoe defect detection method provided in any embodiment corresponding to the present application.

[0174] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0175] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific structure of the ultra-thin high-temperature colored glaze glass with high flatness described above can refer to the corresponding process in the embodiments of the ultra-thin high-temperature colored glaze glass processing method with high flatness described in the above embodiments, and will not be repeated here.

[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A safety shoe defect detection method, characterized in that: The method comprises: Acquire a first-perspective depth image, a second-perspective depth image, and a third-perspective depth image corresponding to the safety shoes to be tested; an image acquisition device corresponding to the first-perspective depth image is arranged relative to an image acquisition device corresponding to the second-perspective depth image, and an image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image; Respectively obtain the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image; Generate a first segmentation point set, a second segmentation point set, and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively; Segmenting the first perspective depth image, the second perspective depth image, and the third perspective depth image according to the first segmentation point set, the second segmentation point set, and the third segmentation point set, respectively, to obtain a plurality of first image blocks, second image blocks, and third image blocks; Respectively obtain the depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks; Extending each of the first image blocks, each of the second image blocks, and each of the third image blocks corresponding to the depth change rate to obtain a first plane image, a second plane image, and a third plane image; Template matching is performed on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection of the safety shoe.

2. The method according to claim 1, characterized in that: The step of respectively acquiring the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image includes: Acquire preset size information corresponding to the safety shoes; the preset size information includes component size information corresponding to a plurality of preset components, and the preset components at least include a sole, an upper, and a toe cap; Establishing a mathematical model corresponding to the preset size information; Based on the random sampling consistency algorithm, the mathematical model is fitted to the first perspective depth image, the second perspective depth image and the third perspective depth image respectively, and the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image and the third perspective depth image are obtained respectively.

3. The method according to claim 1, characterized in that Before respectively acquiring the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, the method further includes: Acquire RGB image information of the safety shoes captured by a preset RGB acquisition device; the RGB image information includes at least color information and texture information; Determining contour information of the safety shoe according to the color information and the texture information; Obtaining a thermal imaging image corresponding to the safety shoes taken by a preset infrared acquisition device; Acquiring temperature change information corresponding to the safety shoes according to the thermal imaging image; The contour information and the temperature change information are mapped to the first-viewing-angle depth image, the second-viewing-angle depth image, and the third-viewing-angle depth image, respectively.

4. The method according to claim 1, characterized in that: The safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively, include: Acquire correlation information and complementarity information between the first-perspective depth image, the second-perspective depth image, and the third-perspective depth image; Establishing a multi-view joint segmentation model according to the correlation information and the complementary information; The safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image and the third perspective depth image are respectively input into the joint segmentation model, and the joint segmentation model generates the first segmentation point set, the second segmentation point set and the third segmentation point set.

5. The method according to claim 1, characterized in that The step of extending each of the first image blocks, each of the second image blocks, and each of the third image blocks according to the depth change rate to obtain a first plane image, a second plane image, and a third plane image includes: Divide each of the first blocks, each of the second blocks, and each of the third blocks according to a plurality of preset areas, and determine the preset area corresponding to each block; the preset area at least includes a sole area, an upper area, and an upper area; According to a preset extension strategy and a depth change rate corresponding to each preset area, each first image block, each second image block, and each third image block corresponding to the preset area are extended to obtain a first plane image, a second plane image, and a third plane image; Among them, the preset extension strategy corresponding to the sole area is linear extension; the preset extension strategy corresponding to the upper area and the vamp area is an extension method based on curve fitting or surface reconstruction.

6. The method according to claim 5, characterized in that The extending each of the first image blocks, each of the second image blocks, and each of the third image blocks corresponding to the preset area according to the preset extension strategy and the depth change rate corresponding to each of the preset areas comprises: Obtaining structural information corresponding to the safety shoes; Obtaining constraint conditions corresponding to each of the preset areas according to the structural information; the constraint conditions at least include the flatness of the sole area, the curvature corresponding to the upper area, and the shape corresponding to the upper area; Each of the first blocks, each of the second blocks, and each of the third blocks corresponding to the preset area are extended according to the preset extension strategy, depth change rate, and constraint conditions corresponding to each of the preset areas.

7. The method according to claim 5, characterized in that Before extending each of the first image blocks, each of the second image blocks, and each of the third image blocks corresponding to the preset area according to the preset extension strategy and the depth change rate corresponding to each of the preset areas, the method further includes: Acquire material properties corresponding to each of the preset areas; the material properties at least include elasticity or hardness; The depth change rate corresponding to each of the first blocks, each of the second blocks, and each of the third blocks in each of the preset areas is corrected according to the material characteristics of each of the preset areas.

8. A safety shoe defect detection device, characterized in that: The device comprises: An image acquisition unit is used to acquire a first-perspective depth image, a second-perspective depth image, and a third-perspective depth image corresponding to the safety shoes to be detected; the image acquisition device corresponding to the first-perspective depth image is arranged relative to the image acquisition device corresponding to the second-perspective depth image, and the image acquisition device corresponding to the third-perspective depth image is arranged perpendicular to the image acquisition device corresponding to the first-perspective depth image; An extreme value acquisition unit, used to respectively acquire the safety shoe contour and depth maximum point corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image; A point set acquisition unit, configured to generate a first segmentation point set, a second segmentation point set, and a third segmentation point set according to the safety shoe contours and depth maximum points corresponding to the first perspective depth image, the second perspective depth image, and the third perspective depth image, respectively; A block acquisition unit, configured to segment the first perspective depth image, the second perspective depth image, and the third perspective depth image according to the first segmentation point set, the second segmentation point set, and the third segmentation point set, respectively, to acquire a plurality of first blocks, second blocks, and third blocks; a change acquisition unit, configured to respectively acquire a depth change rate corresponding to each of the first image blocks, each of the second image blocks, and each of the third image blocks; A picture block extension unit, configured to extend each of the first picture blocks, each of the second picture blocks, and each of the third picture blocks according to the depth change rate, to obtain a first plane image, a second plane image, and a third plane image; The detection completion unit is used to perform template matching on the first plane image, the second plane image and the third plane image respectively with the preset template image corresponding to the safety shoe to complete defect detection on the safety shoe.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method according to any one of claims 1 to 7.

Images