Industrial key point detection method and device based on deep learning and computer equipment
Through the industrial key point detection method based on deep learning, the key point detection model is generated using image data annotation and heat map conversion technology, which solves the problem of insufficient accuracy and generalization capabilities in the existing technology, and realizes the complex industrial environment efficient key point detection.
Patent Information
- Application Number
- CN202411061262.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-07-18
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-05
AI Technical Summary
The existing key point detection technology has problems such as low accuracy, strong dependence on high-quality labeled data, and insufficient generalization capabilities in complex industrial production environments, which limits the development of industrial automation and intelligence.
Using the industrial key point detection method based on deep learning, the key point detection model is generated through image data annotation, labeling point expansion, heat map conversion and model training, and the detection is carried out through the YOLOv8 deep learning model, and the target object is characterized by circular thermal maps. , reduce dependence on high-quality labeled data, and improve detection accuracy and efficiency.
It improves the accuracy and real-time nature of key point detection, reduces dependence on high-quality labeled data, enhances the generalization ability of the algorithm, significantly improves the detection efficiency and accuracy, and is suitable for complex industrial production environments.
Smart Images

Figure CN119048769A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision detection technology, and in particular to an industrial key point detection method, device and computer equipment based on deep learning. Background Art
[0002] Intelligent and automated production processes have become the core driving force for the development of the manufacturing industry. As one of the key technologies for realizing intelligent production, computer vision has greatly promoted the progress of industrial automation through its application in image processing, object detection, and automatic recognition. In particular, key point detection technology plays a vital role in ensuring production quality and improving production efficiency. With the continuous emergence and verification of new technologies, such as deep learning, key point detection technology is also constantly evolving and upgrading to meet the growing needs of industrial production.
[0003] At present, although the existing key point detection technology has achieved certain results, it still faces problems such as low accuracy, strong dependence on a large amount of high-quality labeled data, and insufficient generalization ability. These problems are particularly prominent in complex industrial production environments and have become a key factor restricting the development of industrial automation and intelligence. Therefore, how to optimize the key point detection algorithm, improve its accuracy and real-time performance, reduce dependence on high-quality labeled data, and enhance the generalization ability of the algorithm has become a hot topic and focus of research. Summary of the invention
[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the purpose of the present invention is to propose an industrial key point detection method, device and computer equipment based on deep learning.
[0005] To achieve the above objectives, in a first aspect, an industrial key point detection method based on deep learning according to an embodiment of the present invention includes:
[0006] Image data annotation step: annotate the key points of the image by annotating points;
[0007] Mark point expansion step: each mark point is expanded into a square area centered on the mark point according to the set pixels;
[0008] Heat map conversion step: convert the marked points of each square into the marked points of the circular heat map;
[0009] Model training step: Perform model training on each labeled image to generate a key point detection model;
[0010] Key point detection step: input the detected image into the key point detection model to detect the key points, and output the detected key point coordinates.
[0011] Further, according to an embodiment of the present invention, the outputting of the detected key point coordinates includes:
[0012] Get the information of each output heat map;
[0013] According to the information of each output heat map, the maximum pixel value coordinate point of each output heat map is obtained;
[0014] The maximum pixel value coordinate point of each output heat map is output as the coordinate of each key point.
[0015] Further, according to an embodiment of the present invention, converting the marked points of each square into the marked points of the circular heat map includes:
[0016] Get the center point of the square through the four corners of the square;
[0017] Generate a circular heat map with the center point of the square as the center.
[0018] Further, according to an embodiment of the present invention, the circular heat map marking points are circular map marking points whose image heat values gradually decrease from the center point to the outside.
[0019] Further, according to an embodiment of the present invention, before expanding each marked point into a square area centered on the marked point according to the set pixels, the method further includes:
[0020] Image data conversion step: Convert the annotated json format image data into txt text format image data.
[0021] Further, according to an embodiment of the present invention, the step of expanding each marking point into a square area centered on the marking point according to the set pixels specifically includes expanding each marking point into a square area centered on the marking point according to a size of 50 pixels.
[0022] Further, according to an embodiment of the present invention, the key point detection model is generated by training a segmentation task model of a YOLOv8 deep learning model.
[0023] In a second aspect, an embodiment of the present invention further provides a device for industrial key point detection based on deep learning, comprising:
[0024] Image data annotation module: The image data annotation module is used to annotate key points of the image by annotating points;
[0025] Marking point expansion module: The marking point expansion module is used to expand each marking point into a square area centered on the marking point according to the set pixels;
[0026] Heat map conversion module: The heat map conversion module is used to convert the marked points of each square into the marked points of the circular heat map;
[0027] Model training module: The model training module is used to perform model training on each labeled image to generate a key point detection model;
[0028] Key point detection module: The key point detection module is used to input the detected image into the key point detection model to detect key points, and output the coordinates of the detected key points.
[0029] In a third aspect, a computer device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the industrial key point detection method based on deep learning as described above when executing the computer program.
[0030] According to the industrial key point detection method, device and computer equipment based on deep learning provided by the embodiment of the present invention, the method includes the following steps: image data annotation step: annotating the key points of the image by annotating points; annotated point expansion step: expanding each annotated point into a square area centered on the annotated point according to the set pixels; a heat map conversion step: converting the annotated points of each square into the annotated points of a circular heat map; a model training step: training each marked image with a model to generate a key point detection model; a key point detection step: inputting the detected image into the key point detection model to detect the key points, and detecting the key point coordinates and outputting them. Due to the point annotation method, the annotation is simpler, and the detection accuracy of the key points can be greatly improved by expanding the annotated point pixels and converting the heat map. In addition, the maximum point of the output heat distribution is used as the key point coordinate output during detection, which can greatly improve the detection efficiency.
[0031] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0033] Figure 1 is a flow chart of an industrial key point detection method based on deep learning provided by an embodiment of the present invention;
[0034] Figure 2 is a flow chart of a method for outputting coordinates of key points detected according to an embodiment of the present invention;
[0035] Figure 3 is another flow chart of an industrial key point detection method based on deep learning provided by an embodiment of the present invention;
[0036] Figure 4 It is a structural block diagram of an industrial key point detection device based on deep learning provided by an embodiment of the present invention;
[0037] Figure 5 is a schematic diagram of thermal point conversion provided by an embodiment of the present invention;
[0038] Figure 6 It is an effect diagram of outputting the coordinates of key points detected according to the heat map provided by an embodiment of the present invention;
[0039] Figure 7 It is a structural block diagram of a computer device provided by an embodiment of the present invention.
[0040] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0041] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0042] On the one hand, see Figure 1 , an embodiment of the present invention provides an industrial key point detection method based on deep learning, comprising the steps of:
[0043] Image data labeling step S101: labeling key points of the image by labeling points.
[0044] Marking point expansion step S102: Each marking point is expanded into a square area centered on the marking point according to the set pixels.
[0045] Heat map conversion step S103: converting the marked points of each square into the marked points of a circular heat map.
[0046] Model training step S104: Perform model training on each labeled image to generate a key point detection model.
[0047] Key point detection step S105: input the detected image into the key point detection model to detect key points, and output the coordinates of the detected key points.
[0048] Specifically, in an embodiment of the present invention, the YOLOv8 model can be used to perform deep learning and target detection of image data. In the preparation of training data for the Segment task branch in the YOLOv8 model. In traditional methods, in order to prepare training data for segmentation tasks, it is usually necessary to manually mark and connect each key point around the image to generate a connected domain, which is not only time-consuming and labor-intensive, but also for tasks that require accurate identification of small key points, the accuracy is insufficient and the complexity is extremely high. In order to optimize this process, in the image data annotation step S101, an embodiment of the present invention: the key points of the image are annotated by marking points. In this way, a simpler pose task annotation method in posture detection is adopted. In the pose task annotation method, the annotator only needs to perform simple point annotations at the key point positions, which greatly simplifies the difficulty and time consumption of data preparation.
[0049] After the key points of the image are marked by point marking, the original marked data needs to be converted into a format suitable for YOLOv8 segmentation model (YOLOv8-segment) training through a processing function. In the mark point expansion step S102: each marked point is expanded into a square area centered on the marked point according to the set pixels; wherein in the process of conversion, the marked point is converted into a square roi area centered on the key point. Wherein, the expansion of each marked point into a square area centered on the marked point according to the set pixels is specifically to expand each marked point into a square area centered on the marked point according to the size of 50 pixels. After experimental verification, taking the industrial parts detection project actually applied in the embodiment of the present invention as an example, the segmentation effect of expanding it by 50 pixels is the best. 50 pixels are used to expand the marking points. This approach ensures the size adaptability of the mask area during model training, and at the same time, when it is restored to the original image size for display, it can also ensure the accuracy of the predicted value. The core of this method is to optimize the recognition and segmentation performance of the model for the target area by reasonably setting the size of the prediction box. In the field of target detection and segmentation of deep learning, the size of the prediction box has an important influence on the performance of the model. If the prediction box is too small, some key information may be missed, resulting in inaccurate segmentation; if the prediction box is too large, too much background noise may be introduced, which also affects the accuracy of the segmentation result. Therefore, through appropriate expansion, it can not only ensure that sufficient target information is included, but also suppress the interference of background noise to a certain extent. Through this annotation point expansion step, not only the accuracy of the annotation is maintained, but also the efficiency of training data preparation is significantly improved in subsequent steps, laying a solid foundation for the training of high-precision industrial visual inspection models. The successful application of this method has opened up a new path for data annotation and model training in the field of deep learning, and it also has important reference value for other visual tasks with similar needs.
[0050] See also Figure 5In the above steps, in the annotation point expansion step S102, a formatted square area suitable for YOLOv8 segmentation model (YOLOv8-segment) training is generated. When performing target detection and segmentation based on a deep learning model, the rotation state of the target often poses a challenge to the accurate prediction of the deep learning model, especially when dealing with target objects of various postures in natural scenes. In the traditional polygonal area method, due to the sensitivity of the deep learning model to specific shapes, a rotated or tilted target may cause a decrease in prediction accuracy. In order to further improve the performance of the YOLOv8-Segment model in fine-grained recognition, in the heat map conversion step S103 of the embodiment of the present invention: by converting the annotation points of each square into the annotation points of a circular heat map; that is, by generating a heat map mask of the center point of the square to replace the original method based on polygon conversion into a binary mask. This improved strategy aims to more accurately capture the central area and morphological features of the target object, thereby providing a more accurate segmentation result. Among them, the circular heat map marker is a circular map marker whose image thermal value gradually decreases from the center point to the outside. The core idea is to use heat maps to represent the probability of the existence of target objects, where each point value of the heat map represents the probability that the location is the center of the target. Compared with the traditional polygon method, the heat map method can more intuitively represent the shape and size of the target, especially for target objects of different sizes and irregular shapes, it can provide a more detailed and accurate description.
[0051] See also Figure 5 , wherein the conversion of the annotation points of each square into the annotation points of a circular heat map includes the following steps: first, obtaining the center point of the square through the four corners of the square; and then generating a circular heat map with the center point of the square as the center of the circle. In terms of technical implementation, this method first determines the position of the center point of the target object, then generates square areas of fixed size around these center points, and generates heat maps in these square areas, wherein the heat value at the center point position is the highest and decreases toward the periphery. This mask generation method based on heat maps not only simplifies the annotation process and reduces the complexity of data processing, but also can effectively improve the model's ability to capture target details, thereby improving the segmentation accuracy and the overall performance of the model. Since the circular area is consistent under rotational transformation, it will not change the deep learning model's ability to recognize the target center, thereby ensuring the accuracy and consistency of the prediction results. By adopting the improved strategy of a circular heat map mask, the influence of the target rotation angle on the recognition and segmentation accuracy can be effectively eliminated, thereby further enhancing the prediction accuracy and robustness of the model. Compared with the traditional annotation method based on polygon or square area, the circular heat map mask can provide a more uniform and continuous representation of the target area, which is not affected by the rotation state of the object, thus solving the problems existing in the traditional polygonal mask method.
[0052] After the key points are marked by means of a circular heat map; in the model training step S104: each marked image is subjected to model training to generate a key point detection model; in this step, the key point detection model is trained by inputting the marked image into the deep learning model. Wherein, in an embodiment of the present invention, the key point detection model is trained and generated using the segmentation task model of the YOLOv8 deep learning model, rather than directly using the pose model of the YOLOv8 deep learning model. In an embodiment of the present invention, the yolov8-segment deep learning network model is mainly used, which uses the largest feature map in Head1 as the input of the Mask branch, and obtains a mask_feature of shape: (B, mask_channel, H, W) after being processed by the proto_pred convolutional layer. The prediction branch of the detection box is basically the same as the YoloV5 Head in object detection, except that the parameters for calculating each instance mask are added to the channel of feature_map. The number of parameters is the same as the mask_channel output by proto_pred, so for feature_map of 80X80 / 40X40 / 20X20, the number of channels is: 4+1+num_classes+mask_channel. Get the decoded detection box, and get the final detection box after nms processing. Take the corresponding mask_channel coeffs and mask_feature and multiply them by weight to get the final instance segmentation result.
[0053] After the key point detection model is trained through the yolov8-segment deep learning network model, in an embodiment of the present invention, it is also necessary to deploy and accelerate it through the OpenVINO framework to achieve fast reasoning in different hardware environments.
[0054] OpenVINO supports a wide range of deep learning model acceleration. It can effectively improve the inference efficiency of models on Intel hardware (including CPU, GPU, VPU, etc.) by optimizing graphs of deep learning models, automatically adjusting kernel libraries, and other methods. The specific steps for model acceleration using OpenVINO are as follows:
[0055] Model conversion steps: First, you need to save the trained PyTorch model as a .pt weight file, and then use the ONNX tool to convert it to the ONNX format. ONNX is an open model representation format that allows developers to easily migrate and deploy models between different deep learning frameworks.
[0056] OpenVINO optimization steps: The converted ONNX format model is then optimized using OpenVINO's ModelOptimizer. At this stage, the Model Optimizer will perform in-depth analysis of the model, perform graph transformations, prune redundant operations, and optimize the hierarchy to improve model execution efficiency.
[0057] Reasoning acceleration step: The optimized model is deployed and executed on the target device through OpenVINO's Inference Engine. The Inference Engine can intelligently call the most optimized calculation path suitable for the current hardware, and can achieve efficient model reasoning whether on the CPU or GPU.
[0058] Through the above process, not only can the inference speed of 50-80 milliseconds per image be achieved on the CPU, but with the support of GPU, the inference performance of the model can be reduced to less than 50 milliseconds, greatly improving the response speed and real-time processing capability of the model in practical applications. This is of great significance for application scenarios that require fast recognition and segmentation (such as real-time video analysis, robot vision, etc.).
[0059] After the detection model is deployed to the detection device, the key point detection model can be used to detect the key points of the detected object. In the key point detection step S105: the detected image is input into the key point detection model for key point detection, and the key point coordinates are detected and output. In the instance segmentation key point detection strategy based on heat map, the output channel of the model is designed to be 4+1+num_classes+mask_channel to fully meet the needs of segmentation target area, category prediction and key point detection. In this design, the original segmentation model output not only includes the segmentation results for each target area, but also includes the category prediction of the corresponding target.
[0060] See also Figure 2 and Figure 6 In the processing of outputting the coordinates of the detected key points, two different implementation methods can be adopted, as follows.
[0061] See also Figure 2 and Figure 6 In one embodiment of the present invention, Figure 2 As shown in , the output of the detected key point coordinates includes the steps of:
[0062] S1051a. Obtain information of each output heat map.
[0063] S1052a. Obtain the maximum pixel value coordinate point of each output heat map according to the information of each output heat map.
[0064] S1053a. Output the maximum pixel value coordinate point of each output heat map as the coordinate of each key point.
[0065] Specifically, in step S1051a, the information of each output heat map is obtained; since the detection output information of the yolov8-segment deep learning network model includes the heat map information of the prediction box and the prediction area. The position information of the key point can be determined based on these two types of information. In the detection of key points through heat map information, the detection of key points is achieved by finding the maximum value point of the heat distribution in the target area after segmentation, which is an important step in determining the coordinates of the target key point. In step S1052a, the maximum pixel value coordinate point of each output heat map is obtained according to the information of each output heat map; since the area segmented by segment is not exactly the same as the actual marked circular mask, and even the general prediction area cannot be a regular circle, it is only necessary to find the position with the largest value in the heat map as the key point position. In practical applications, the accuracy of the key point coordinates can be ensured by searching for the maximum value point of heat in each area through pixel loops. After obtaining the coordinates of the maximum value point of the heat map, in step S1053a, the maximum pixel value coordinate point of each output heat map is output as the coordinates of each key point. As shown in FIG. Figure 7 In the figure, A1, A2 and A3 are the key point outputs for detection.
[0066] Since 50 pixels are used to expand the marking points in the annotation point expansion step S102, this approach ensures the size adaptability of the mask area during model training, and also ensures the accuracy of the predicted value when it is restored to the original image size for display. The core of this method is to optimize the model's recognition and segmentation performance of the target area by reasonably setting the size of the prediction box. In the field of target detection and segmentation of deep learning, the size of the prediction box has an important impact on the performance of the model. If the prediction box is too small, some key information may be missed, resulting in inaccurate segmentation; and if the prediction box is too large, too much background noise may be introduced, which also affects the accuracy of the segmentation result. Therefore, through appropriate expansion, it can not only ensure that sufficient target information is included, but also suppress the interference of background noise to a certain extent. Specifically, through the expansion of 50 pixels, not only the model can learn the shape and feature information of the target more comprehensively, but also help to divide the target boundary more accurately during instance segmentation, thereby improving the accuracy of segmentation. In addition, when the segmentation result is restored to the original image for display, such a prediction box size setting is also conducive to maintaining the consistency between the segmented area and the actual target, ensuring the accurate reflection of the prediction result. In this way, it can be distinguished that in the key point detection stage, the deep learning model can output a prediction box containing the key points.
[0067] See also Figure 3 In one embodiment of the present invention, before expanding each annotation point into a square area centered on the annotation point according to the set pixels, the method also includes the following steps: image data conversion step S2025: converting the annotated json format image data into txt text format image data.
[0068] The designed script automatically converts the json files of these point annotations into the label format suitable for the Segment segmentation task. The first 5 data formats of the point annotation json after conversion to txt are the upper left and lower right of the workpiece type processing part box (x1, y1, x2, y2); starting from the 6th value, every 3 values represent the (x, y) coordinates of the key points and their visibility. It is only necessary to classify the key points according to the workpiece category and index value to which they belong, and use their coordinates (x, y) as the center coordinates of the square, widen 50 pixels up and down and left and right, and generate the four vertex coordinates of the square (x1, y1, x2, y2, x3, y3, x4, y4).
[0069] Ginseng Figure 4 On the other hand, the present invention also provides an industrial key point detection device based on deep learning, including: an image data annotation module, a annotation point expansion module, a heat map conversion module, a model training module and a key point detection module, wherein the image data annotation module is used to annotate the key points of the image by annotating points.
[0070] The marking point expansion module is used to expand each marking point into a square area centered on the marking point according to the set pixels.
[0071] The heat map conversion module is used to convert the marked points of each square into the marked points of the circular heat map.
[0072] The model training module is used to perform model training on each labeled image to generate a key point detection model.
[0073] The key point detection module is used to input the detected image into the key point detection model to detect key points, and output the detected key point coordinates.
[0074] See also Figure 7 On the other hand, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned industrial key point detection method based on deep learning when executing the computer program.
[0075] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the computer device.
[0076] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the figure is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the computer device may also include an input / output device, a network access device, a bus, etc.
[0077] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete preset hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0078] The memory may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The memory may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the memory may include both an internal storage unit and an external storage device of the computer device. The memory is used to store the computer program and other programs and data required by the computer device. The memory may also be used to temporarily store data that has been output or is to be output.
[0079] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the industrial key point detection method based on deep learning as described above is implemented.
[0080] The computer program may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above method embodiments may be implemented. The computer program includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0081] It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.
[0082] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0083] The steps in the method of the embodiment of the present invention can be adjusted in order, combined or deleted according to actual needs.
[0084] The modules or units in the system of the embodiment of the present invention may be combined, divided or deleted according to actual needs.
[0085] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic preset hardware, or in a combination of computer software and electronic preset hardware. Whether these functions are performed in preset hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0086] In the embodiments provided by the present invention, it should be understood that the disclosed device / computer equipment 600 and method can be implemented in other ways. For example, the device / computer equipment 600 embodiment described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0087] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A method for detecting industrial key points based on deep learning, characterized in that: include: Image data annotation step: annotate the key points of the image by annotating points; Mark point expansion step: each mark point is expanded into a square area centered on the mark point according to the set pixels; Heat map conversion step: convert the marked points of each square into the marked points of the circular heat map; Model training step: Perform model training on each labeled image to generate a key point detection model; Key point detection step: input the detected image into the key point detection model to detect the key points, and output the detected key point coordinates.
2. The industrial key point detection method based on deep learning according to claim 1 is characterized in that: The detected key point coordinate output includes: Get the information of each output heat map; According to the information of each output heat map, the maximum pixel value coordinate point of each output heat map is obtained; The maximum pixel value coordinate point of each output heat map is output as the coordinate of each key point.
3. The industrial key point detection method based on deep learning according to claim 1 or 2, characterized in that: The method of converting the marked points of each square into the marked points of the circular heat map includes: Get the center point of the square through the four corners of the square; Generate a circular heat map with the center point of the square as the center.
4. The industrial key point detection method based on deep learning according to claim 3 is characterized in that: The circular heat map marking points are circular map marking points whose image heat values gradually decrease from the center point to the outside.
5. The industrial key point detection method based on deep learning according to claim 1 or 2, characterized in that: Before expanding each marked point into a square area centered on the marked point according to the set pixels, the method further includes: Image data conversion step: Convert the annotated json format image data into txt text format image data.
6. The industrial key point detection method based on deep learning according to claim 1 or 2, characterized in that: The method of expanding each marked point into a square area centered on the marked point according to the set pixels is specifically as follows: Each marked point is expanded into a square area centered on the marked point in size of 50 pixels.
7. The industrial key point detection method based on deep learning according to claim 1 or 2, characterized in that: The key point detection model is generated by training the segmentation task model of the YOLOv8 deep learning model.
8. An industrial key point detection device based on deep learning, characterized in that: include: Image data annotation module: The image data annotation module is used to annotate key points of the image by annotating points; Marking point expansion module: The marking point expansion module is used to expand each marking point into a square area centered on the marking point according to the set pixels; Heat map conversion module: The heat map conversion module is used to convert the marked points of each square into the marked points of the circular heat map; Model training module: The model training module is used to perform model training on each labeled image to generate a key point detection model; Key point detection module: The key point detection module is used to input the detected image into the key point detection model to detect key points, and output the coordinates of the detected key points.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the deep learning-based industrial key point detection method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Image processing method and device and computer readable storage medium
CN111598899A
Key point detection model training method, key point detection method and related device
CN117934827A
Image annotation method, apparatus, device and medium
WO2021217543A1
Cited By
Industrial key point data cleaning and abnormal point eliminating method and related device
CN120804526A
Method for cleaning and removing abnormal points of industrial key point data and related device
CN120804526B
