Skid deformation monitoring method and device

Abstract

The invention discloses a skid deformation monitoring method and device. The method comprises the following steps: acquiring training image data, and establishing a data set according to the training image data; training the basic convolutional neural network model by using the training data set to obtain an application convolutional neural network model, and testing the application convolutional neural network model by using the test data set to obtain a prediction area of the test data set; comparing the intersection-to-union ratio with a standard intersection-to-union ratio, and comparing the center distance with a standard center distance according to a comparison condition; the method comprises the following steps: acquiring real-time production data in a production field, inputting the real-time production data into a final application convolutional neural network model to obtain a model prediction region, calculating a region center point according to the model prediction region, calculating distances among four vertexes to obtain four-point distances, and calculating a mean square error according to the four-point distances and a standard distance. And comparing the mean square deviation with a standard mean square deviation. According to the invention, the problem of inaccurate monitoring of the skid deformation degree in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment monitoring, and in particular to a method and device for monitoring the deformation of a sled. Background Art

[0002] In the related art, the automatic detection method for the deformation degree of the sled generally adopts a detection method combining a mechanical and a photoelectric switch.

[0003] First, fix the slide as a whole, and then the detection device falls along the slide rail. If the deformation of the slide is normal, all the pins will smoothly enter the mechanical sleeve and lift the slider, so that the slider senses the proximity switch and the detection passes. If the deformation of the slide is too large, the pins cannot enter the corresponding mechanical sleeve, and thus cannot sense the proximity switch, and the detection fails.

[0004] This method has extremely high requirements on the positioning accuracy of the slide in the front, back, left and right directions, and cannot eliminate the relative errors of all pins after the slide is positioned offset.

[0005] In the related art, the technical problem of inaccurate monitoring of the deformation degree of the skid has not yet been effectively solved. Summary of the invention

[0006] The main purpose of the present application is to provide a method and device for monitoring the deformation of a sled, so as to at least solve the problem of inaccurate monitoring of the deformation degree of the sled in the related art.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for monitoring the deformation of a skid is provided. The method includes: obtaining training image data, forming a data set according to the training image data, wherein the data set includes a training data set and a test data set; using the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, using the test data set to test the applied convolutional neural network model to obtain a predicted area of ​​the test data set; obtaining the intersection-and-union ratio and center-to-center distance between the predicted area and the annotated area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center-to-center distance with the standard center-to-center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center-to-center distance is less than the standard center-to-center distance, including: when the comparison condition is met, determining that the applied convolutional neural network model is the final application Use a convolutional neural network model; when the comparison conditions are not met, perform incremental training on the applied convolutional neural network model; obtain real-time production data at the production site, input the real-time production data into the final applied convolutional neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between the four vertices to obtain the four-point distance, calculate the mean square deviation according to the four-point distance and the standard distance, and compare the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, mark the sled deformation degree as too large; if the mean square deviation is less than or equal to the standard mean square deviation, mark the sled deformation degree as normal.

[0008] Optionally, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0009] Optionally, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0010] Optionally, calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the ordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0011] Optionally, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0012] Optionally, calculate the center spacing using the formula ,in, is the center distance.

[0013] Optionally, calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0014] According to another aspect of the present application, a monitoring device for the deformation of a sled is provided. The device includes: an acquisition unit, used to acquire training image data, and to form a data set according to the training image data, wherein the data set includes a training data set and a test data set; a training unit, used to train a basic convolutional neural network model using the training data set to obtain an applied convolutional neural network model, and to test the applied convolutional neural network model using the test data set to obtain a predicted area of ​​the test data set; a comparison unit, used to obtain an intersection-and-union ratio and a center distance between the predicted area and the labeled area, and to compare the intersection-and-union ratio with a standard intersection-and-union ratio, and to compare the center distance with a standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determining that the applied convolutional neural network The network model is the final application convolutional neural network model; when the comparison conditions are not met, the application convolutional neural network model is incrementally trained; a judgment unit is used to obtain real-time production data at the production site, input the real-time production data into the final application convolutional neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between the four vertices to obtain the four-point distance, calculate the mean square deviation according to the four-point distance and the standard distance, and compare the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, it is marked that the sled deformation degree is too large; if the mean square deviation is less than or equal to the standard mean square deviation, it is marked that the sled deformation degree is normal.

[0015] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for monitoring the deformation of a skid.

[0016] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for monitoring the deformation of a skid for executing any one of the items.

[0017] Through the present application, the following steps are adopted: obtaining training image data, forming a data set according to the training image data, wherein the data set includes a training data set and a test data set; using the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, using the test data set to test the applied convolutional neural network model to obtain a predicted area of ​​the test data set; obtaining the intersection-over-union ratio and the center spacing between the predicted area and the labeled area, comparing the intersection-over-union ratio with the standard intersection-over-union ratio, and comparing the center spacing with the standard center spacing according to a comparison condition, wherein the comparison condition is that the intersection-over-union ratio is greater than the standard intersection-over-union ratio and the center spacing is less than the standard center spacing, including: when the comparison condition is met, determining that the applied convolutional neural network model is the final applied convolutional neural network model; when the comparison condition is not met, comparing the applied convolutional neural network model. Incremental training; obtain real-time production data at the production site, input the real-time production data into the final application convolutional neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between the four vertices to get the four-point distance, calculate the mean square error according to the four-point distance and the standard distance, and compare the mean square error with the standard mean square error, where the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square error is greater than the standard mean square error, mark the sled deformation degree as too large; if the mean square error is less than or equal to the standard mean square error, mark the sled deformation degree as normal, which solves the problem of inaccurate monitoring of the sled deformation degree in related technologies, thereby achieving higher reliability product quality detection and real-time monitoring, and effectively avoiding the subjectivity and individual differences in the manual inspection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 is a flow chart of a method for monitoring sled deformation provided according to an embodiment of the present application;

[0020] Figure 2 It is a flow chart of an overall solution provided according to an embodiment of the present application;

[0021] Figure 3 It is a structural block diagram of a sled deformation monitoring device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] As introduced in the background technology, in the prior art, observing the deformation of the sled by mechanical jacking cannot accurately monitor the degree of deformation of the sled, takes up a large beat, cannot record the sled number, and is not convenient for subsequent tracking. In order to solve the problem of inaccurate monitoring of the deformation degree of the sled in the related art, the embodiments of the present application provide a method and device for monitoring the deformation of the sled.

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0027] In this embodiment, a method for monitoring the deformation of a skid is provided that runs on a mobile terminal, a computer terminal or a similar computing device. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] Figure 1 FIG. 1 is a flow chart of a method for monitoring the deformation of a sled according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0029] Step S101, obtaining training image data, and forming a data set according to the training image data, wherein the data set includes a training data set and a test data set;

[0030] Specifically, the image data of the collected training data set are generally images of known sled deformation and images of known sled non-deformation, and the training data set is added to the neural network model for training. Then a batch of image data for testing is obtained.

[0031] Step S102, using the training data set to train the basic convolutional neural network model to obtain the applied convolutional neural network model, and using the test data set to test the applied convolutional neural network model to obtain the prediction area of ​​the test data set;

[0032] Specifically, the convolutional neural network model is tested to obtain a prediction area, which is the main area for determining whether the skid is deformed in the image.

[0033] Step S103, obtaining the intersection-and-union ratio and the center distance between the predicted area and the marked area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determining that the applied convolutional neural network model is the final applied convolutional neural network model; when the comparison condition is not met, performing incremental training on the applied convolutional neural network model;

[0034] Specifically, the intersection-over-unit ratio and the center-to-center distance are obtained by testing the predicted area and the labeled area using the convolutional neural network model. The labeled area is set in advance. The labeled area is an object with obvious features that the model manually marks on the picture. The purpose of the labeled area is to allow the model to recognize similar objects in unfamiliar pictures after learning.

[0035] Step S204, acquiring real-time production data at the production site, inputting the real-time production data into the final application convolutional neural network model, obtaining the model prediction area, calculating the area center point according to the model prediction area, calculating the distance between the four vertices to obtain the four-point distance, calculating the mean square deviation according to the four-point distance and the standard distance, and comparing the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, marking the sled deformation degree is too large; if the mean square deviation is less than or equal to the standard mean square deviation, marking the sled deformation degree is normal.

[0036] Specifically, in actual application, an image of the sled is taken at the production site, and the image data is put into the final application convolutional neural network model, and the model prediction area is obtained using the final application convolutional neural network model. The distance between the four points is calculated. There are a total of six line segments between the four points with an accuracy of 0.1 mm. Distance from standard Calculate mean square error , and then compare the mean square error and standard deviation The standard mean square error is calculated based on the average of the normal image data according to the degree of deformation. Finally, the convolutional neural network model is used to predict the four center points based on the four pictures. , the four center points can be divided into 6 groups: ( )( )( )( )( )( ), according to the distance calculation formula, we get the 6 side lengths The four-point distance and sled deformation judgment results will be displayed on the data image.

[0037] Through the deformation monitoring method based on visual recognition in this embodiment, through high-precision image acquisition and processing technology, the tiny deformation of the sled can be monitored in real time, and its detection accuracy can reach the sub-micron level, breaking through the physical limitations of the human eye. The visual recognition system can achieve more reliable product quality detection and real-time monitoring, and effectively avoid the subjectivity and individual differences in the manual detection process. The visual recognition system can be flexibly configured and adjusted according to actual needs to adapt to different monitoring needs.

[0038] Reference Figure 2 , after obtaining the image data, a data set is formed, and then the image data in the data set is processed, the model is trained according to the data set, the trained model is tested, the intersection ratio and the center distance are compared, and the model can be deployed after the comparison meets the conditions. If the conditions are not met, incremental training is performed until the conditions are met; after the model is deployed, four pin images of the sled and the sled number in the production process are obtained, and real-time data is obtained during the production process to obtain the predicted area. The center point of the area is calculated according to the model predicted area, the four-point distance is calculated, and then the four-point distance, the detection result and the sled number are displayed on the interface, and the mean square error of the four-point distance and the standard distance is compared. If the conditions are not met, modification suggestions are put forward, and then model training is performed.

[0039] In an optional embodiment, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0040] Specifically, by manually annotating with rectangular boxes and labels, the position and type of the sled pins can be accurately located, which helps improve the accuracy and efficiency of subsequent image recognition or classification tasks. The information included in the labels, such as the category, center point coordinates, rectangular box height and width, provides rich feature data for model training. Converting the training images into three-dimensional arrays can be easily processed using deep learning frameworks. Deep learning models usually require input data to have fixed dimensions and shapes, and the three-dimensional array format can meet this requirement. Adjusting the dimension order of the three-dimensional array and adjusting the number of channels to the first position can ensure that the input data meets the input format requirements of the deep learning model. Unifying the resolution size of all training images can ensure that the model has consistent performance when processing images of different sizes. This helps the model better learn and identify the features of the sled pins. Unifying the resolution can also reduce the time and computing resource consumption of image preprocessing and improve the efficiency of the entire image processing process.

[0041] In addition, the slide pin images are divided into other types such as crossbeam and latch, and the data set is screened in the total data set according to the slide pin categories. The number of corresponding images of each category is not less than 800, and the data set is divided into a training data set and a test data set according to a 4:1 ratio. The training data set and the test data set are randomly allocated.

[0042] In an optional embodiment, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0043] Specifically, by continuously recording the accuracy, recall, and average precision of each iteration, the model's training progress and performance changes can be monitored in real time. This helps to promptly detect problems in the training process, such as overfitting, underfitting, or data imbalance. Accuracy, recall, and average precision are important indicators for evaluating the performance of classification models. They reflect the model's ability to identify positive samples and the accuracy of the overall classification, respectively. When these indicators all reach a high level (such as above 0.99), the model can be considered to have excellent classification performance.

[0044] In addition, requiring the evaluation data to remain at a high level after 50 iterations can ensure that the selected model has sufficient stability and robustness. This helps to reduce the volatility and uncertainty of the model in practical applications and improve the overall reliability and accuracy of the system. When the model reaches the standard for training completion, the obtained model parameters are fully optimized and verified. These parameters can accurately reflect the inherent laws and characteristics of the data, providing a solid foundation for subsequent model deployment and application. By setting clear training completion standards and iteration number limits, unnecessary overtraining and waste of resources can be avoided. This helps to save computing resources and time costs and improve the efficiency of model development. Models that have been fully trained and verified can show better performance in practical applications. This helps to improve the recognition accuracy, response speed and user experience of the system, and provide strong technical support for the development and application of related fields.

[0045] In an optional embodiment, the range of the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the vertical coordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0046] In an optional embodiment, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0047] In an optional embodiment, the center distance is calculated using the formula: ,in, is the center distance.

[0048] In an optional embodiment, the mean square error is calculated using the following formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0049] The important invention points of this application are:

[0050] 1. Visual recognition technology: This method is based on visual recognition technology, which can monitor the deformation of the sled in real time through high-precision image acquisition and processing. Visual recognition technology includes but is not limited to industrial cameras, convolutional neural network models and other technologies, which can generate two-dimensional images of the sled pins and achieve accurate measurement of the sled deformation.

[0051] 2. Deformation monitoring: The system can monitor the deformation of the sled in real time. By comparing the standard state of the sled with the real-time monitoring state, the deformation problem of the sled can be discovered in time.

[0052] 3. Automation and intelligence: The entire monitoring and alarm process is automated and intelligent, without the need for human intervention. The system can automatically collect data, process data, determine deformation conditions, and trigger alarm functions.

[0053] The technical effects achieved by the present invention are:

[0054] 1. High-precision monitoring: The deformation monitoring method based on visual recognition can monitor the tiny deformation of the sled in real time through high-precision image acquisition and processing technology. Its detection accuracy can reach the sub-micron level, breaking through the physical limitations of the human eye. Although mechanical lifting can lift or move the sled, it often cannot provide such high-precision data in deformation monitoring.

[0055] 2. Higher reliability: The visual recognition system can achieve more reliable product quality detection and real-time monitoring, effectively avoiding the subjectivity and individual differences in the manual detection process. Mechanical lifting may be affected by many factors during operation, such as mechanical wear, operating errors, etc., which may affect the reliability of its monitoring results.

[0056] 3. High flexibility: The visual recognition system can be flexibly configured and adjusted according to actual needs to adapt to different monitoring needs. Mechanical lifting is relatively poor in flexibility and usually needs to be customized according to specific application scenarios.

[0057] The embodiment of the present application also provides a monitoring device for the deformation of a sled. It should be noted that the monitoring device for the deformation of a sled in the embodiment of the present application can be used to execute a monitoring method for the deformation of a sled provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0058] The following is an introduction to a sled deformation monitoring device provided in an embodiment of the present application.

[0059] Figure 3 is a structural block diagram of a sled deformation monitoring device according to an embodiment of the present application. Figure 3As shown, the device includes: an acquisition unit 301, used to acquire training image data, and to form a data set according to the training image data, wherein the data set includes a training data set and a test data set; a training unit 302, used to train a basic convolutional neural network model using the training data set to obtain an applied convolutional neural network model, and to test the applied convolutional neural network model using the test data set to obtain a predicted area of ​​the test data set; a comparison unit 303, used to obtain an intersection-and-union ratio and a center distance between the predicted area and the labeled area, compare the intersection-and-union ratio with the standard intersection-and-union ratio, and compare the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determining that the applied convolutional neural network model is applied; The convolution neural network model is the final application convolution neural network model; when the comparison condition is not met, the application convolution neural network model is incrementally trained; the judgment unit 304 is used to obtain real-time production data at the production site, input the real-time production data into the final application convolution neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between the four vertices to obtain the four-point distance, calculate the mean square deviation according to the four-point distance and the standard distance, and compare the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, it is marked that the sled deformation degree is too large; if the mean square deviation is less than or equal to the standard mean square deviation, it is marked that the sled deformation degree is normal.

[0060] As an optional solution, the acquisition unit 301 includes: a labeling subunit, which is used to select training images according to the type of pins of the sled, and manually label them using rectangular boxes and labels, wherein the labels include categories, horizontal coordinates of the center points, vertical coordinates of the center points, heights of the rectangular boxes, and widths of the rectangular boxes; a conversion subunit, which is used to read the training images and convert the training images into three-dimensional arrays; an adjustment subunit, which is used to adjust the dimensional order of the three-dimensional array and adjust the number of channels to the first position; a normalization subunit, which is used to divide the pixel values ​​of the training images by 255 to normalize the training images; and a unification subunit, which is used to unify the resolution sizes of all training images.

[0061] As an optional solution, the training unit 302 includes: a recording subunit, which is used to record the evaluation data of the basic convolutional neural network model after each iterative training, and uses accuracy, recall rate, and average precision as evaluation data; a determination subunit, which is used to determine that the training of the application convolutional neural network model is completed and obtain the model parameters after the evaluation data are all greater than 0.99 and maintained for 50 iterations.

[0062] As an optional solution, the comparison unit 303 includes: a first calculation subunit, which is used to calculate the range of the marked area, and the calculation formula is: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the second calculation subunit is used to calculate the area of ​​the marked area, and the calculation formula is: ,in, is the area of ​​the marked area; the third calculation subunit is used to calculate the range of the predicted area, and the calculation formula is: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the ordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular frame of the prediction area; the fourth calculation subunit is used to calculate the area of ​​the prediction area, and the calculation formula is: ,in, is the area of ​​the prediction region.

[0063] As an optional solution, the comparison unit 303 includes: a fifth calculation subunit, which is used to calculate the intersection area of ​​the predicted area and the marked area, and the calculation formula is: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; the sixth calculation subunit is used to calculate the area of ​​the intersection area, and the calculation formula is: ,in, is the area of ​​the intersection, for The seventh calculation subunit is used to calculate the intersection-and-union ratio, and the calculation formula is: ,in, It is the intersection and comparison.

[0064] As an optional solution, the comparison unit 303 includes: an eighth calculation subunit, which is used to calculate the center distance, and the calculation formula is: ,in, is the center distance.

[0065] As an optional solution, the determination unit 304 includes: a ninth calculation subunit, which is used to calculate the mean square error, and the calculation formula is: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0066] The monitoring device for the deformation of a sled includes a processor and a memory, and the acquisition unit 301 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0067] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set to solve C technical problems by adjusting kernel parameters.

[0068] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0069] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for monitoring the deformation of a skid.

[0070] Specifically, a method for monitoring the deformation of a sled includes:

[0071] Acquire training image data, and form a data set according to the training image data, wherein the data set includes a training data set and a test data set; use the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, and use the test data set to test the applied convolutional neural network model to obtain a predicted area of ​​the test data set; obtain an intersection-over-union ratio and a center distance between the predicted area and the annotated area, compare the intersection-over-union ratio with the standard intersection-over-union ratio, and compare the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-over-union ratio is greater than the standard intersection-over-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determine that the applied convolutional neural network model is the final applied convolution Neural network model; when the comparison conditions are not met, incremental training is performed on the applied convolutional neural network model; real-time production data is obtained at the production site, the real-time production data is input into the final applied convolutional neural network model, the model prediction area is obtained, the center point of the area is calculated according to the model prediction area, the distance between the four vertices is calculated to obtain the four-point distance, the mean square deviation is calculated according to the four-point distance and the standard distance, and the mean square deviation is compared with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, the sled deformation degree is marked as too large; if the mean square deviation is less than or equal to the standard mean square deviation, the sled deformation degree is marked as normal.

[0072] Optionally, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0073] Optionally, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0074] Optionally, calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the ordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0075] Optionally, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0076] Optionally, calculate the center spacing using the formula ,in, is the center distance.

[0077] Optionally, calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0078] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for monitoring the deformation of a sled is executed when the program is run.

[0079] Specifically, a method for monitoring the deformation of a sled includes:

[0080] Acquire training image data, and form a data set according to the training image data, wherein the data set includes a training data set and a test data set; use the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, and use the test data set to test the applied convolutional neural network model to obtain a predicted area of ​​the test data set; obtain an intersection-over-union ratio and a center distance between the predicted area and the annotated area, compare the intersection-over-union ratio with the standard intersection-over-union ratio, and compare the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-over-union ratio is greater than the standard intersection-over-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determine that the applied convolutional neural network model is the final applied convolution Neural network model; when the comparison conditions are not met, incremental training is performed on the applied convolutional neural network model; real-time production data is obtained at the production site, the real-time production data is input into the final applied convolutional neural network model, the model prediction area is obtained, the center point of the area is calculated according to the model prediction area, the distance between the four vertices is calculated to obtain the four-point distance, the mean square deviation is calculated according to the four-point distance and the standard distance, and the mean square deviation is compared with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, the sled deformation degree is marked as too large; if the mean square deviation is less than or equal to the standard mean square deviation, the sled deformation degree is marked as normal.

[0081] Optionally, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0082] Optionally, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0083] Optionally, calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the vertical coordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0084] Optionally, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection region, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0085] Optionally, calculate the center spacing using the formula ,in, is the center distance.

[0086] Optionally, calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0087] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are implemented: obtaining training image data, forming a data set according to the training image data, wherein the data set includes a training data set and a test data set; training a basic convolutional neural network model using the training data set to obtain an applied convolutional neural network model, and testing the applied convolutional neural network model using the test data set to obtain a predicted area of ​​the test data set; obtaining an intersection-over-union ratio and a center distance between the predicted area and the labeled area, comparing the intersection-over-union ratio with a standard intersection-over-union ratio, and comparing the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-over-union ratio is greater than the standard intersection-over-union ratio and the center distance is less than the standard center distance, Including: when the comparison conditions are met, determining that the applied convolutional neural network model is the final applied convolutional neural network model; when the comparison conditions are not met, incrementally training the applied convolutional neural network model; obtaining real-time production data at the production site, inputting the real-time production data into the final applied convolutional neural network model, obtaining the model prediction area, calculating the center point of the area according to the model prediction area, calculating the distance between the four vertices to obtain the four-point distance, calculating the mean square error according to the four-point distance and the standard distance, comparing the mean square error with the standard mean square error, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square error is greater than the standard mean square error, marking the sled deformation degree is too large; if the mean square error is less than or equal to the standard mean square error, marking the sled deformation degree is normal. The device in this article can be a server, PC, PAD, mobile phone, etc.

[0088] Optionally, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0089] Optionally, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0090] Optionally, calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the vertical coordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0091] Optionally, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection region, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0092] Optionally, calculate the center spacing using the formula ,in, is the center distance.

[0093] Optionally, calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0094] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with at least the following method steps: obtaining training image data, forming a data set based on the training image data, wherein the data set includes a training data set and a test data set; using the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, and using the test data set to test the applied convolutional neural network model to obtain a predicted area of ​​the test data set; obtaining an intersection-over-union ratio and a center distance between the predicted area and the labeled area, comparing the intersection-over-union ratio with a standard intersection-over-union ratio, and comparing the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-over-union ratio is greater than the standard intersection-over-union ratio and the center distance is less than the standard center distance, including: when the intersection-over-union ratio is satisfied, When the comparison conditions are met, the applied convolutional neural network model is determined to be the final applied convolutional neural network model; when the comparison conditions are not met, the applied convolutional neural network model is incrementally trained; real-time production data is obtained at the production site, the real-time production data is input into the final applied convolutional neural network model, the model prediction area is obtained, the center point of the area is calculated according to the model prediction area, the distance between the four vertices is calculated to obtain the four-point distance, the mean square deviation is calculated according to the four-point distance and the standard distance, and the mean square deviation is compared with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, the sled deformation degree is marked as too large; if the mean square deviation is less than or equal to the standard mean square deviation, the sled deformation degree is marked as normal.

[0095] Optionally, a training image is selected according to the type of pins of the sled, and is manually annotated using a rectangular frame and labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; the training image is read and converted into a three-dimensional array; the dimensional order of the three-dimensional array is adjusted, and the number of channels is adjusted to the first position; the pixel values ​​of the training images are divided by 255 to normalize the training images; and the resolution size of all training images is unified.

[0096] Optionally, after each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; after the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

[0097] Optionally, calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the vertical coordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; the area of ​​the marked area is calculated using the following formula: ,in, is the area of ​​the marked area; the range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the vertical coordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular box of the prediction area; the area of ​​the prediction area is calculated using the following formula: ,in, is the area of ​​the prediction region.

[0098] Optionally, the intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; calculate the area of ​​the intersection area, the calculation formula is: ,in, is the area of ​​the intersection region, for The absolute value of; calculate the intersection ratio, the calculation formula is ,in, It is the intersection and comparison.

[0099] Optionally, calculate the center spacing using the formula ,in, is the center distance.

[0100] Optionally, calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

[0101] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0107] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0108] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0109] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0110] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for monitoring the deformation of a sled, characterized in that: include: Acquire training image data, and form a data set according to the training image data, wherein the data set includes a training data set and a test data set; Using the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, and using the test data set to test the applied convolutional neural network model to obtain a prediction region of the test data set; Obtaining the intersection-and-union ratio and the center distance between the predicted area and the annotated area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determining that the applied convolutional neural network model is the final applied convolutional neural network model; when the comparison condition is not met, performing incremental training on the applied convolutional neural network model; Acquire real-time production data at the production site, input the real-time production data into the final application convolutional neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between the four vertices to obtain the four-point distance, calculate the mean square deviation according to the four-point distance and the standard distance, and compare the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of the rectangle formed by the four vertices, including: if the mean square deviation is greater than the standard mean square deviation, mark the sled deformation degree as too large; if the mean square deviation is less than or equal to the standard mean square deviation, mark the sled deformation degree as normal.

2. The method according to claim 1, characterized in that Acquire training image data, and form a data set according to the training image data, wherein the data set includes a training data set and a test data set, including: Select training images according to the type of pins of the sled, use rectangular frames and manually annotate them with labels, wherein the labels include category, horizontal coordinate of the center point, vertical coordinate of the center point, height of the rectangular frame, and width of the rectangular frame; Reading the training image, and converting the training image into a three-dimensional array; Adjust the dimensional order of the three-dimensional array and adjust the number of channels to the first place; The pixel values ​​of the training images are all divided by 255 to normalize the training images; The resolution size of all the training images is unified.

3. The method according to claim 1, characterized in that Using the training data set to train a basic convolutional neural network model to obtain an applied convolutional neural network model, using the test data set to test the applied convolutional neural network model to obtain a prediction region of the test data set, including: After each iterative training, the evaluation data of the basic convolutional neural network model is recorded, and the accuracy, recall rate, and average precision are used as the evaluation data; After the evaluation data are all greater than 0.99 and maintained for 50 iterations, it is determined that the training of the applied convolutional neural network model is completed, and the model parameters are obtained.

4. The method according to claim 1, characterized in that Obtaining the intersection-and-union ratio and the center distance between the predicted area and the marked area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to the comparison condition, including: Calculate the range of the marked area using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the marked area, is the horizontal coordinate of the lower right vertex of the marked area, is the ordinate of the upper left vertex of the marked area, is the ordinate of the lower right vertex of the marked area, is the horizontal coordinate of the center point of the marked area, is the ordinate of the center point of the marked area, is the width of the rectangular frame of the marked area, is the height of the rectangular frame of the marked area; Calculate the area of ​​the marked area using the following formula: ,in, is the area of ​​the marked area; The range of the predicted area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the prediction area, is the horizontal coordinate of the lower right vertex of the prediction area, is the ordinate of the upper left vertex of the prediction area, is the ordinate of the lower right vertex of the prediction area, is the horizontal coordinate of the center point of the prediction area, is the ordinate of the center point of the prediction area, is the width of the prediction area rectangle, is the height of the rectangular frame of the prediction area; The area of ​​the predicted region is calculated using the following formula: ,in, is the area of ​​the prediction region.

5. The method according to claim 4, characterized in that Obtaining the intersection-and-union ratio and the center distance between the predicted area and the marked area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to the comparison condition, including: The intersection area of ​​the predicted area and the marked area is calculated using the following formula: , , , ,in, is the horizontal coordinate of the upper left vertex of the intersection area, is the horizontal coordinate of the lower right vertex of the intersection area, is the ordinate of the upper left vertex of the intersection area, is the ordinate of the lower right vertex of the intersection area; The area of ​​the intersection region is calculated using the following formula: ,in, is the area of ​​the intersection region, for The absolute value of Calculate the intersection-and-union ratio, the calculation formula is: ,in, is the intersection-and-union ratio.

6. The method according to claim 4, characterized in that Obtaining the intersection-and-union ratio and the center distance between the predicted area and the marked area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to the comparison condition, including: Calculate the center distance, the calculation formula is: ,in, is the center distance.

7. The method according to claim 1, characterized in that Acquire real-time production data at the production site, input the real-time production data into the final application convolutional neural network model, obtain the model prediction area, calculate the area center point according to the model prediction area, calculate the distance between four vertices to obtain the four-point distance, calculate the mean square error according to the four-point distance and the standard distance, and compare the mean square error with the standard mean square error, including: Calculate the mean square error using the formula: ,in, is the mean square error, is the distance between the four points, is the standard distance.

8. A device for monitoring the deformation of a sled, characterized in that: include: An acquisition unit, used to acquire training image data, and to form a data set according to the training image data, wherein the data set includes a training data set and a test data set; A training unit, configured to train a basic convolutional neural network model using the training data set to obtain an applied convolutional neural network model, and to test the applied convolutional neural network model using the test data set to obtain a prediction region of the test data set; A comparison unit, used for obtaining the intersection-and-union ratio and the center distance between the predicted area and the annotated area, comparing the intersection-and-union ratio with the standard intersection-and-union ratio, and comparing the center distance with the standard center distance according to a comparison condition, wherein the comparison condition is that the intersection-and-union ratio is greater than the standard intersection-and-union ratio and the center distance is less than the standard center distance, including: when the comparison condition is met, determining that the applied convolutional neural network model is the final applied convolutional neural network model; when the comparison condition is not met, performing incremental training on the applied convolutional neural network model; A determination unit is used to obtain real-time production data at the production site, input the real-time production data into the final application convolutional neural network model, obtain a model prediction area, calculate a region center point according to the model prediction area, calculate the distance between four vertices to obtain a four-point distance, calculate a mean square deviation according to the four-point distance and a standard distance, and compare the mean square deviation with the standard mean square deviation, wherein the four-point distance is the four sides and two diagonals of a rectangle formed by four vertices, including: if the mean square deviation is greater than the standard mean square deviation, marking the sled deformation degree as excessive; if the mean square deviation is less than or equal to the standard mean square deviation, marking the sled deformation degree as normal.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute a method for monitoring the deformation of a skid as claimed in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for monitoring the deformation of a skid as described in any one of claims 1 to 7.

Images