Full-automatic crane pipe state recognition method and system based on visual image recognition

Abstract

The invention discloses a full-automatic crane pipe state recognition method and system based on visual image recognition, relates to the field of industrial production, and solves the problem that an existing full-automatic crane pipe state recognition method is poor in recognition effect. The method comprises the following steps: S1, creating an appearance image recognition model to recognize a plurality of crane pipe appearance images to obtain crane pipe appearance type division data, S2, respectively obtaining a crane pipe pressure period monitoring coefficient and a crane pipe temperature period monitoring coefficient corresponding to a full-automatic crane pipe to obtain crane pipe physical monitoring data, and performing physical monitoring on the crane pipe; and S3, work recognition is conducted on the full-automatic crane pipe according to the crane pipe appearance type division data and the crane pipe physical monitoring data, and work state early warning is issued according to the recognition result. The pertinence, comprehensiveness and accuracy of crane pipe state recognition can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial production and relates to visual image recognition technology, in particular to a full-automatic crane state recognition method and system based on visual image recognition. Background Art

[0002] The existing fully automatic crane status recognition method has the following specific defects when identifying the crane: 1. The data processing speed of the existing fully automatic crane status identification method is not fast enough to meet the needs of real-time monitoring, which easily leads to delayed response and failure to identify potential faults or risks in a timely manner; 2. The existing fully automatic crane status recognition method usually uses monitoring data of a specific period of time to perform phased status recognition, which can easily lead to inaccurate recognition results.

[0003] To this end, we propose a fully automatic crane status recognition method and system based on visual image recognition. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a fully automatic crane status recognition method and system based on visual image recognition, and the present invention aims to improve the accuracy and pertinence of the fully automatic crane status recognition method.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: a fully automatic crane state recognition method based on visual image recognition, comprising the following specific steps: Step S1: acquiring appearance images of the crane pipe at multiple different positions to obtain multiple crane pipe appearance images, creating an appearance image recognition model to recognize the multiple crane pipe appearance images, and classifying the fully automatic crane pipe into a first appearance type crane pipe and a second appearance type crane pipe according to the recognition results to obtain crane pipe appearance type classification data; Step S2: performing periodic pressure monitoring and periodic temperature monitoring on the crane of the second appearance type in working state, and respectively obtaining the crane pressure periodic monitoring coefficient and the crane temperature periodic monitoring coefficient by analyzing the monitoring results, and obtaining the crane physical monitoring data; Step S3: Identify the working state of the fully automatic crane pipe according to the crane pipe appearance type classification data and the crane pipe physical monitoring data, and issue a working state warning according to the identification result.

[0006] A fully automatic crane status recognition system based on visual image recognition, comprising: Visual image module: used to obtain appearance images of crane pipes at multiple different positions, obtain multiple crane pipe appearance images, create an appearance image recognition model to recognize multiple crane pipe appearance images, and classify the fully automatic crane pipes into first appearance type crane pipes and second appearance type crane pipes according to the recognition results, and obtain crane pipe appearance type classification data; Physical data module: used to perform periodic pressure monitoring and periodic temperature monitoring on the second appearance type crane in working state, and obtain the crane pressure periodic monitoring coefficient and the crane temperature periodic monitoring coefficient respectively by analyzing the monitoring results, and obtain the crane physical monitoring data; Status recognition module: used to identify the working state of the fully automatic crane according to the crane appearance type classification data and the crane physical monitoring data, and issue working state warnings based on the recognition results.

[0007] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. The present invention obtains appearance images of a plurality of different positions of a crane pipe, obtains a plurality of crane pipe appearance images, creates an appearance image recognition model to recognize the plurality of crane pipe appearance images, divides the fully automatic crane pipe into a first appearance type crane pipe and a second appearance type crane pipe according to the recognition results, and performs physical data detection on the second appearance type crane pipe, thereby improving the pertinence of crane pipe status monitoring; 2. The present invention can effectively improve the comprehensiveness of the monitoring process and the accuracy of the monitoring results by performing periodic pressure monitoring and periodic temperature monitoring on the fully automatic crane pipe. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0009] Figure 1 It is a diagram of the implementation steps of the present invention; Figure 2 It is the overall system block diagram of the present invention. DETAILED DESCRIPTION

[0010] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] Embodiment 1

[0012] See also Figure 1 The present invention provides a technical solution: a fully automatic crane state recognition method based on visual image recognition, comprising the following specific steps: Step S1: acquiring appearance images of the crane pipe at multiple different positions to obtain multiple crane pipe appearance images, creating an appearance image recognition model to recognize the multiple crane pipe appearance images, and classifying the fully automatic crane pipe into a first appearance type crane pipe and a second appearance type crane pipe according to the recognition results to obtain crane pipe appearance type classification data; The step S1 further includes the following specific steps: Step S11: marking a plurality of characteristic monitoring locations in the fully automatic crane in a working state, and naming the marked plurality of characteristic monitoring locations as the first characteristic monitoring location to the ath characteristic monitoring location respectively; Step S12: acquiring images of the first characteristic monitoring part to the ath characteristic monitoring part respectively by an image acquisition device to obtain images of the first characteristic part to the ath characteristic part; Step S13: obtaining a network image of each characteristic monitoring part of the fully automatic crane pipe by using data crawler technology to obtain a plurality of crane pipe network images; Step S14: using a plurality of crane pipe network images to create a crane pipe appearance recognition model; The step S14 further includes the following specific steps: Step S141: dividing a plurality of crane pipe network images into first-type crane pipe images and second-type crane pipe images by manual identification and manual labeling to obtain crane pipe network image labeling data; Step S142: dividing the crane network image labeling data into a crane image training set and a crane image test set according to the image training test ratio; Step S143: creating an image recognition model through an existing artificial intelligence platform, and training the image recognition model using a crane image training set, until each medical crane image in the crane image training set has trained the image recognition model once; Step S144: Use the crane image test set to test the image recognition model and obtain the recognition accuracy. When the recognition accuracy is greater than or equal to the target recognition accuracy, the image recognition model training is completed and the crane appearance recognition model is obtained. When the recognition accuracy is less than the target recognition accuracy, continue to use the crane image training set to train the image recognition model until the recognition accuracy is greater than or equal to the target recognition accuracy. Step S15: using the crane appearance recognition model to perform image recognition on the first characteristic part image to the ath characteristic part image respectively to obtain crane appearance type classification data; If any one of the first characteristic part image to the ath characteristic part image is a first type of crane tube image, then the fully automatic crane tube is determined to be a first appearance type crane tube; If the first characteristic part image to the ath characteristic part image are all second type crane images, then the fully automatic crane is determined to be a second appearance type crane; Step S2: performing periodic pressure monitoring and periodic temperature monitoring on the crane of the second appearance type in working state, and respectively obtaining the crane pressure periodic monitoring coefficient and the crane temperature periodic monitoring coefficient by analyzing the monitoring results, and obtaining the crane physical monitoring data; The step S2 further includes the following specific steps: Step S21: acquiring crane appearance type classification data, and acquiring cranes of the second appearance type respectively according to the crane appearance type classification data; Step S22: During the period of physical data monitoring of the crane pipe of the first appearance type, mark the time value corresponding to the current moment as the first physical monitoring time point, mark a second physical monitoring time point in the period before the first physical monitoring time point, and mark the period between the first physical monitoring time point and the second physical monitoring time point as the physical data real-time monitoring period; Step S23: performing periodic pressure monitoring on the second appearance type crane in the physical data real-time monitoring period to obtain the crane pressure period monitoring coefficient; The step S23 further includes the following specific steps: Step S231: arranging a plurality of pressure monitoring sensors inside the crane tube of the second appearance type; Step S232: marking a number of pressure monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and naming the marked pressure monitoring time points as T1 pressure monitoring time point to Tb pressure monitoring time point in chronological order; Step S233: Obtain T1 monitoring pressure value and T1 pressure monitoring coefficient; The step S233 further includes the following specific steps: Step S2331: respectively obtaining the monitoring pressure value corresponding to each pressure monitoring sensor at the pressure monitoring point T1 to obtain a plurality of monitoring pressure values; Step S2332: Calculate the average of the obtained multiple monitoring pressure values ​​to obtain the T1 monitoring pressure value; Step S2333: performing variance calculation on the obtained multiple monitoring pressure values ​​to obtain T1 monitoring pressure variance; Step S2334: Calculate the T1 monitoring pressure value and the T1 monitoring pressure variance to obtain the pressure monitoring coefficient corresponding to the pressure monitoring sensor at the T1 pressure monitoring point, and name it T1 pressure monitoring coefficient; Calculate the T1 pressure monitoring coefficient, the specific formula is as follows: ; Among them, Tyx1 is the T1 pressure monitoring coefficient, Typ1 is the T1 monitoring pressure value, and Tfc1 is the T1 monitoring pressure variance; Step S234: respectively acquiring the pressure monitoring coefficients corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb, and obtaining the pressure monitoring coefficients T2 to Tb; Step S235: respectively acquiring the monitoring pressure values ​​corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb, and obtaining the monitoring pressure values ​​T2 to Tb; Step S236: obtaining a reference pressure value of the crane of the second appearance type within a physical data real-time monitoring period, and obtaining a period reference pressure value; Step S237: Obtain the difference between the T1 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T1 pressure monitoring deviation, obtain the difference between the T2 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T2 pressure monitoring deviation, and so on, obtain the difference between the Tb monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the Tb pressure monitoring deviation; Step S238: Calculate the average of the pressure monitoring deviation from T1 to Tb to obtain the period average pressure monitoring deviation; Step S239: Calculate the periodic average pressure deviation and the pressure monitoring coefficient T1 to the pressure monitoring coefficient Tb to obtain the crane pressure periodic monitoring coefficient; The crane pressure cycle monitoring coefficient is calculated, and the specific formula is as follows: ; Among them, Yzj is the crane pressure cycle monitoring coefficient, Ypp is the cycle average pressure deviation, Tyxi is the Ti pressure monitoring coefficient, Tyx(i-1) is the Ti-1 pressure monitoring coefficient, and b is the quantity value corresponding to the pressure monitoring time point; Step S24: performing periodic temperature monitoring on the second appearance type crane tube in the physical data real-time monitoring period to obtain the crane tube temperature periodic monitoring coefficient; The step S24 further includes the following specific steps: Step S241: multiple temperature monitoring sensors are arranged inside the crane tube of the second appearance type; Step S242: marking a number of temperature monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and naming the marked temperature monitoring time points as W1 temperature monitoring time point to Wc temperature monitoring time point in chronological order; Step S243: Obtain W1 monitoring temperature value and W1 temperature monitoring coefficient; The step S243 further includes the following specific steps: Step S2431: respectively obtain the monitoring temperature value corresponding to each temperature monitoring sensor at the W1 temperature monitoring point to obtain multiple monitoring temperature values, average the obtained multiple monitoring temperature values ​​to obtain the W1 monitoring temperature value, and perform variance calculation on the obtained multiple monitoring temperature values ​​to obtain the W1 monitoring temperature variance; Step S2432: Calculate the W1 monitoring temperature value and the W1 monitoring temperature variance to obtain the temperature monitoring coefficient corresponding to the temperature monitoring sensor at the W1 temperature monitoring point, and name it W1 temperature monitoring coefficient; Calculate the W1 temperature monitoring coefficient, the specific formula is as follows: ; Among them, Wyx1 is the W1 temperature monitoring coefficient, Wyp1 is the W1 monitoring temperature value, and Wfc1 is the W1 monitoring temperature variance; Step S244: respectively acquiring the temperature monitoring coefficients corresponding to the temperature monitoring time point W2 to the temperature monitoring time point Wc, and obtaining the temperature monitoring coefficients W2 to Wc; Step S245: respectively acquiring the monitoring temperature values ​​corresponding to the temperature monitoring time point W2 to the temperature monitoring time point Wc, and obtaining the monitoring temperature values ​​W2 to Wc; Step S246: obtaining a reference temperature value of the crane of the second appearance type within a physical data real-time monitoring period to obtain a period reference temperature value; Step S247: Obtain the difference between the W1 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W1 temperature monitoring deviation, obtain the difference between the W2 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W2 temperature monitoring deviation, and so on, obtain the difference between the Wc monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the Wc temperature monitoring deviation; Step S248: Calculate the average of the W1 temperature monitoring deviation to the Wc temperature monitoring deviation to obtain the period average temperature monitoring deviation; Step S249: Calculate the periodic average temperature deviation and the W1 temperature monitoring coefficient to the Wc temperature monitoring coefficient to obtain the crane tube temperature periodic monitoring coefficient; The crane pipe temperature cycle monitoring coefficient is calculated, and the specific formula is as follows: ; Among them, Wzj is the crane temperature period monitoring coefficient, Wpp is the period average temperature deviation, Wyxi is the Wi temperature monitoring coefficient, Wyx(i-1) is the Wi-1 temperature monitoring coefficient, and c is the quantity value corresponding to the temperature monitoring time point; Step S25: defining the crane temperature periodic monitoring coefficient and the crane pressure periodic monitoring coefficient as the crane physical monitoring data; Step S3: identifying the working state of the fully automatic crane pipe according to the crane pipe appearance type classification data and the crane pipe physical monitoring data, and issuing a working state warning according to the identification result; The step S3 further includes the following specific steps: Step S31: acquiring the crane appearance type classification data, and acquiring the first appearance type crane and the second appearance type crane according to the crane appearance type classification data; Step S32: if the automated crane is a crane of the first appearance type, determining that the automated crane is in an abnormal working state, and issuing a crane working state warning; Step S33: if the automated crane is of the second appearance type, the working state of the automated crane is determined according to the physical monitoring data of the crane, and an abnormal state warning is issued to the automated crane according to the determination result; The step S33 further includes the following specific steps: Step S331: obtaining the crane pipe physical monitoring data, and obtaining the crane pipe temperature cycle monitoring coefficient and the crane pipe pressure cycle monitoring coefficient respectively according to the crane pipe physical monitoring data; Step S332: obtaining a crane state judgment coefficient by calculating a crane temperature periodic monitoring coefficient and a crane pressure periodic monitoring coefficient; The crane status judgment coefficient is calculated, and the specific formula is as follows:

[0013] Among them, Hpd is the crane state judgment coefficient, Wzj is the crane temperature cycle monitoring coefficient, and Yzj is the crane pressure cycle monitoring coefficient; Step S333: respectively obtaining a threshold value of a periodic monitoring coefficient of a temperature of a crane and a threshold value of a periodic monitoring coefficient of a pressure of a crane; Step S334: The crane state judgment coefficient threshold is obtained by calculating the crane temperature cycle monitoring coefficient threshold and the crane pressure cycle monitoring coefficient threshold; Step S335: if the crane state judgment coefficient is greater than or equal to the crane state judgment coefficient threshold, it is determined that the automated crane working state is abnormal, and a crane working state warning is issued; Step S336: If the crane state judgment coefficient is less than the crane state judgment coefficient threshold, it is determined that the automated crane working state is normal, and no crane working state warning is issued.

[0014] In this application, if corresponding calculation formulas appear, the above calculation formulas are all dimensionless and take their numerical calculations. The weight coefficients, proportional coefficients and other coefficients in the formulas are set to a result value obtained by quantifying each parameter. The size of the weight coefficient and the proportional coefficient can be determined as long as it does not affect the proportional relationship between the parameter and the result value.

[0015] Embodiment 2

[0016] See also Figure 2 , based on another concept of the same invention, a fully automatic crane pipe state recognition system based on visual image recognition is now proposed, including a visual image module, a physical data module, a state recognition module and a server, wherein the visual image module, the physical data module and the state recognition module are respectively connected to the server, and the server controls the visual image module, the physical data module and the state recognition module respectively; The visual image module acquires appearance images of the crane pipe at multiple different positions, obtains multiple crane pipe appearance images, creates an appearance image recognition model to recognize the multiple crane pipe appearance images, and divides the fully automatic crane pipe into a first appearance type crane pipe and a second appearance type crane pipe according to the recognition results, to obtain crane pipe appearance type classification data; The details are as follows: Marking a number of characteristic monitoring locations in the fully automatic crane in a working state, and naming the marked number of characteristic monitoring locations as the first characteristic monitoring location to the ath characteristic monitoring location respectively; It should be noted here that: In this application, a referred to herein is a quantity value corresponding to a characteristic monitoring part, and a is an integer greater than 0; In the present application, the first characteristic monitoring part involved here may be a conveying pipeline, the second characteristic monitoring part may be a flange connection part, and the third characteristic monitoring part may be a lifting device; By using an image acquisition device, images of the first characteristic monitoring part to the ath characteristic monitoring part are acquired respectively, so as to obtain images of the first characteristic part to the ath characteristic part; The network image of each characteristic monitoring part of the fully automatic crane pipe is obtained by using data crawler technology to obtain multiple crane pipe network images; Using multiple crane network images to create a crane appearance recognition model; The details are as follows: By manually identifying and manually marking a plurality of crane pipe network images, the plurality of crane pipe network images are divided into first-type crane pipe images and second-type crane pipe images to obtain crane pipe network image marking data; It should be noted here that: In the present application, the first type of crane tube image designed here is an image corresponding to a part with abnormal appearance, and the second type of crane tube image designed here is an image corresponding to a part with normal appearance; The crane network image labeling data is divided into a crane image training set and a crane image test set according to the image training and testing ratio; It should be noted here that: In the present application, the image training test ratio involved here is specifically 7:3, that is, the ratio of the number of images in the Crane image training set and the Crane image test set is 7:3; Create an image recognition model through the existing artificial intelligence platform, and use the crane image training set to train the image recognition model until each medical crane image in the crane image training set is used to train the image recognition model; It should be noted here that: In this application, the artificial intelligence platform designed here is specifically TensorFlow; The image recognition model is tested using the crane image test set, and the recognition accuracy is obtained. When the recognition accuracy is greater than or equal to the target recognition accuracy, the image recognition model training is completed, and the crane appearance recognition model is obtained. When the recognition accuracy is less than the target recognition accuracy, the image recognition model is continuously trained using the crane image training set until the recognition accuracy is greater than or equal to the target recognition accuracy. Use the crane pipe appearance recognition model to perform image recognition on the first feature part image to the ath feature part image respectively to obtain crane pipe appearance type classification data; If any one of the first characteristic part image to the ath characteristic part image is a first type of crane tube image, then the fully automatic crane tube is determined to be a first appearance type crane tube; If the first characteristic part image to the ath characteristic part image are all second type crane images, then the fully automatic crane is determined to be a second appearance type crane; The visual image module acquires the data of the crane pipe appearance classification and transmits it to the physical data module and the state recognition module; The physical data module performs periodic pressure monitoring and periodic temperature monitoring on the second appearance type crane pipe in working state, and obtains the crane pipe pressure periodic monitoring coefficient and the crane pipe temperature periodic monitoring coefficient respectively by analyzing the monitoring results, and obtains the crane pipe physical monitoring data; The details are as follows: Acquire the crane pipe appearance type classification data, and acquire the crane pipes of the second appearance type respectively according to the crane pipe appearance type classification data; During the period of physical data monitoring of the crane pipe of the first appearance type, the time value corresponding to the current moment is marked as the first physical monitoring time point, a second physical monitoring time point is marked in the period before the first physical monitoring time point, and the period between the first physical monitoring time point and the second physical monitoring time point is marked as the physical data real-time monitoring cycle; It should be noted here that: In the present application, as the time value corresponding to the current moment changes, the first physical monitoring time point and the second physical monitoring time point also change accordingly, thereby achieving dynamic update of the physical data time monitoring period; Perform periodic pressure monitoring on the second appearance type crane in the physical data real-time monitoring period to obtain the crane pressure period monitoring coefficient; The details are as follows: A plurality of pressure monitoring sensors are arranged inside the crane pipe of the second appearance type; Mark a number of pressure monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and name the marked pressure monitoring time points as T1 pressure monitoring time point to Tb pressure monitoring time point in chronological order; It should be noted here that: In this application, T referred to herein is an identifier corresponding to a pressure monitoring time point, b is a quantity value corresponding to a pressure monitoring time point, and b is an integer greater than 0; Obtaining the monitoring pressure value corresponding to each pressure monitoring sensor at the T1 pressure monitoring point respectively, obtaining multiple monitoring pressure values, calculating the average of the obtained multiple monitoring pressure values, obtaining the T1 monitoring pressure value, and calculating the variance of the obtained multiple monitoring pressure values, obtaining the T1 monitoring pressure variance; The pressure monitoring coefficient corresponding to the pressure monitoring sensor at the T1 pressure monitoring point is obtained by calculating the T1 monitoring pressure value and the T1 monitoring pressure variance, and is named the T1 pressure monitoring coefficient; Calculate the T1 pressure monitoring coefficient, the specific formula is as follows: ; Among them, Tyx1 is the T1 pressure monitoring coefficient, Typ1 is the T1 monitoring pressure value, and Tfc1 is the T1 monitoring pressure variance; The pressure monitoring coefficients corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb are obtained respectively, and the pressure monitoring coefficients from T2 to Tb are obtained; The monitoring pressure values ​​corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb are obtained respectively, and the monitoring pressure values ​​T2 to Tb are obtained; Obtain a reference pressure value of the crane of the second appearance type within a physical data real-time monitoring period to obtain a period reference pressure value; It should be noted here that: In the present application, if the substances transported by the second appearance type crane pipe in the real-time monitoring period of physical data are the same and the external transport conditions are the same, then the periodic reference pressure values ​​in the second appearance type crane pipe are also the same; In the present application, the external transport conditions involved herein include but are not limited to ambient temperature, ambient humidity and material transport speed.

[0017] Obtain the difference between the T1 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T1 pressure monitoring deviation, obtain the difference between the T2 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T2 pressure monitoring deviation, and so on, obtain the difference between the Tb monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the Tb pressure monitoring deviation; The pressure monitoring deviation of T1 to the pressure monitoring deviation of Tb are averaged to obtain the period average pressure monitoring deviation; The crane pressure cycle monitoring coefficient is obtained by calculating the cycle average pressure deviation and the T1 pressure monitoring coefficient to the Tb pressure monitoring coefficient; The crane pressure cycle monitoring coefficient is calculated, and the specific formula is as follows: ; Among them, Yzj is the crane pressure cycle monitoring coefficient, Ypp is the cycle average pressure deviation, Tyxi is the Ti pressure monitoring coefficient, Tyx(i-1) is the Ti-1 pressure monitoring coefficient, and b is the quantity value corresponding to the pressure monitoring time point; It should be noted here that: The Ti pressure monitoring coefficient involved here can be any pressure monitoring coefficient from the T1 pressure monitoring coefficient to the Tb pressure monitoring coefficient; Perform periodic temperature monitoring on the second appearance type crane tube in the physical data real-time monitoring period to obtain the crane tube temperature periodic monitoring coefficient; The details are as follows: A plurality of temperature monitoring sensors are arranged inside the crane tube of the second appearance type; Mark a number of temperature monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and the marked temperature monitoring time points are named W1 temperature monitoring time point to Wc temperature monitoring time point in chronological order; It should be noted here that: In this application, W referred to herein is an identifier corresponding to a temperature monitoring time point, c is a quantity value corresponding to a temperature monitoring time point, and c is an integer greater than 0; Obtain the monitoring temperature value corresponding to each temperature monitoring sensor at the W1 temperature monitoring point respectively, obtain multiple monitoring temperature values, calculate the average of the obtained multiple monitoring temperature values, obtain the W1 monitoring temperature value, and calculate the variance of the obtained multiple monitoring temperature values ​​to obtain the W1 monitoring temperature variance; The W1 monitoring temperature value and the W1 monitoring temperature variance are calculated to obtain the temperature monitoring coefficient corresponding to the temperature monitoring sensor at the W1 temperature monitoring point, and named it the W1 temperature monitoring coefficient; Calculate the W1 temperature monitoring coefficient, the specific formula is as follows: ; Among them, Wyx1 is the W1 temperature monitoring coefficient, Wyp1 is the W1 monitoring temperature value, and Wfc1 is the W1 monitoring temperature variance; The temperature monitoring coefficients corresponding to the W2 temperature monitoring time point to the Wc temperature monitoring time point are obtained respectively, and the W2 temperature monitoring coefficient to the Wc temperature monitoring coefficient are obtained; Acquire the monitoring temperature values ​​corresponding to the W2 temperature monitoring time point to the Wc temperature monitoring time point respectively, and obtain the W2 monitoring temperature value to the Wc monitoring temperature value; Obtain a reference temperature value of the crane pipe of the second appearance type within a physical data real-time monitoring period to obtain a period reference temperature value; It should be noted here that: In the present application, if the substances transported by the second appearance type crane pipe in the physical data real-time monitoring period are the same and the external transport conditions are the same, then the periodic reference temperature values ​​in the second appearance type crane pipe are also the same; In the present application, the external transport conditions involved herein include but are not limited to ambient temperature, ambient humidity and material transport speed.

[0018] Obtain the difference between the W1 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W1 temperature monitoring deviation, obtain the difference between the W2 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W2 temperature monitoring deviation, and so on, obtain the difference between the Wc monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the Wc temperature monitoring deviation; The W1 temperature monitoring deviation and the Wc temperature monitoring deviation are averaged to obtain the period average temperature monitoring deviation. The periodic average temperature deviation and the W1 temperature monitoring coefficient to the Wc temperature monitoring coefficient are calculated to obtain the crane tube temperature periodic monitoring coefficient; The crane pipe temperature cycle monitoring coefficient is calculated, and the specific formula is as follows: ; Among them, Wzj is the crane temperature period monitoring coefficient, Wpp is the period average temperature deviation, Wyxi is the Wi temperature monitoring coefficient, Wyx(i-1) is the Wi-1 temperature monitoring coefficient, and c is the quantity value corresponding to the temperature monitoring time point; It should be noted here that: The Wi temperature monitoring coefficient involved here can be any temperature monitoring coefficient from the W1 temperature monitoring coefficient to the Wc temperature monitoring coefficient; The crane pipe temperature cycle monitoring coefficient and the crane pipe pressure cycle monitoring coefficient are defined as the crane pipe physical monitoring data; The physical data module acquires the physical monitoring data of the crane pipe and transmits it to the state identification module; The status recognition module identifies the working state of the fully automatic crane according to the crane appearance type classification data and the crane physical monitoring data, and issues a working state warning based on the recognition results; The details are as follows: Acquire the crane pipe appearance type classification data, and acquire the first appearance type crane pipe and the second appearance type crane pipe respectively according to the crane pipe appearance type classification data; If the automated crane is a crane of the first appearance type, it is determined that the automated crane is in an abnormal working state, and a crane working state warning is issued; If the automated crane is of the second appearance type, the working status of the automated crane is determined based on the physical monitoring data of the crane, and an abnormal status warning is issued to the automated crane based on the determination result; The details are as follows: Obtain the crane pipe physical monitoring data, and obtain the crane pipe temperature cycle monitoring coefficient and the crane pipe pressure cycle monitoring coefficient respectively according to the crane pipe physical monitoring data; The crane pipe temperature cycle monitoring coefficient and the crane pipe pressure cycle monitoring coefficient are calculated to obtain the crane pipe state judgment coefficient; The crane status judgment coefficient is calculated, and the specific formula is as follows:

[0019] Among them, Hpd is the crane state judgment coefficient, Wzj is the crane temperature cycle monitoring coefficient, and Yzj is the crane pressure cycle monitoring coefficient; Obtain the threshold of the periodic monitoring coefficient of the crane temperature and the threshold of the periodic monitoring coefficient of the crane pressure respectively; It should be noted here that: In the present application, the crane pipe temperature cycle monitoring coefficient threshold and the crane pipe pressure cycle monitoring coefficient threshold involved here are respectively the maximum crane pipe temperature cycle monitoring coefficient and the maximum crane pipe pressure cycle monitoring coefficient corresponding to the automated crane pipe in normal working state; The crane pipe temperature cycle monitoring coefficient threshold and the crane pipe pressure cycle monitoring coefficient threshold are calculated to obtain the crane pipe state judgment coefficient threshold; The threshold value of the crane status judgment coefficient is calculated, and the specific formula is as follows:

[0020] Among them, Hpdy is the threshold value of the crane state judgment coefficient, Wzjy is the threshold value of the crane temperature cycle monitoring coefficient, and Yzjy is the threshold value of the crane pressure cycle monitoring coefficient; If the crane control state judgment coefficient is greater than or equal to the crane control state judgment coefficient threshold, the automated crane control working state is judged to be abnormal, and a crane control working state warning is issued; If the crane control state judgment coefficient is less than the crane control state judgment coefficient threshold, the automated crane control working state is judged to be normal, and no crane control working state warning is issued.

[0021] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A fully automatic crane status recognition method based on visual image recognition, characterized in that: include: Step S1: acquiring appearance images of the crane pipe at multiple different positions to obtain multiple crane pipe appearance images, creating an appearance image recognition model to recognize the multiple crane pipe appearance images, and classifying the fully automatic crane pipe into a first appearance type crane pipe and a second appearance type crane pipe according to the recognition results to obtain crane pipe appearance type classification data; Step S2: performing periodic pressure monitoring and periodic temperature monitoring on the crane of the second appearance type in working state, and respectively obtaining the crane pressure periodic monitoring coefficient and the crane temperature periodic monitoring coefficient by analyzing the monitoring results, and obtaining the crane physical monitoring data; Step S3: Identify the working state of the fully automatic crane pipe according to the crane pipe appearance type classification data and the crane pipe physical monitoring data, and issue a working state warning according to the identification result.

2. According to the method of claim 1, the method is characterized in that: The step S1 further includes the following specific steps: Step S11: marking a plurality of characteristic monitoring locations in the fully automatic crane in a working state, and naming the marked plurality of characteristic monitoring locations as the first characteristic monitoring location to the ath characteristic monitoring location respectively; Step S12: acquiring images of the first characteristic monitoring part to the ath characteristic monitoring part respectively by an image acquisition device to obtain images of the first characteristic part to the ath characteristic part; Step S13: obtaining a network image of each characteristic monitoring part of the fully automatic crane pipe by using data crawler technology to obtain a plurality of crane pipe network images; Step S14: using a plurality of crane pipe network images to create a crane pipe appearance recognition model; Step S15: using the crane appearance recognition model to perform image recognition on the first characteristic part image to the ath characteristic part image respectively to obtain crane appearance type classification data; If any one of the first characteristic part image to the ath characteristic part image is a first type of crane tube image, then the fully automatic crane tube is determined to be a first appearance type crane tube; If the first characteristic part image to the ath characteristic part image are all second type crane images, then the fully automatic crane is determined to be a second appearance type crane.

3. The method for fully automatic crane status recognition based on visual image recognition according to claim 2 is characterized in that: The step S14 further includes the following specific steps: Step S141: dividing a plurality of crane pipe network images into first-type crane pipe images and second-type crane pipe images by manual identification and manual labeling to obtain crane pipe network image labeling data; Step S142: dividing the crane network image labeling data into a crane image training set and a crane image test set according to the image training test ratio; Step S143: creating an image recognition model through an existing artificial intelligence platform, and training the image recognition model using a crane tube image training set; Step S144: Use the crane image test set to test the image recognition model and obtain the recognition accuracy. When the recognition accuracy is greater than or equal to the target recognition accuracy, the image recognition model training is completed and the crane appearance recognition model is obtained. When the recognition accuracy is less than the target recognition accuracy, continue to use the crane image training set to train the image recognition model until the recognition accuracy is greater than or equal to the target recognition accuracy.

4. The method for fully automatic crane status recognition based on visual image recognition according to claim 1 is characterized in that: The step S2 further includes the following specific steps: Step S21: acquiring crane appearance type classification data, and acquiring cranes of the second appearance type respectively according to the crane appearance type classification data; Step S22: marking a physical data real-time monitoring cycle during the period of monitoring the physical data of the crane tube of the first appearance type; Step S23: performing periodic pressure monitoring on the second appearance type crane in the physical data real-time monitoring period to obtain the crane pressure period monitoring coefficient; Step S24: performing periodic temperature monitoring on the second appearance type crane tube in the physical data real-time monitoring period to obtain the crane tube temperature period monitoring coefficient; Step S25: defining the crane temperature periodic monitoring coefficient and the crane pressure periodic monitoring coefficient as the crane physical monitoring data.

5. The method for fully automatic crane status recognition based on visual image recognition according to claim 4 is characterized in that: The step S23 further includes the following specific steps: Step S231: arranging a plurality of pressure monitoring sensors inside the crane tube of the second appearance type; Step S232: marking a number of pressure monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and naming the marked pressure monitoring time points as T1 pressure monitoring time point to Tb pressure monitoring time point in chronological order; Step S233: Obtain T1 monitoring pressure value and T1 pressure monitoring coefficient; Step S234: respectively acquiring the pressure monitoring coefficients corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb, and obtaining the pressure monitoring coefficients T2 to Tb; Step S235: respectively acquiring the monitoring pressure values ​​corresponding to the pressure monitoring time point T2 to the pressure monitoring time point Tb, and obtaining the monitoring pressure values ​​T2 to Tb; Step S236: obtaining a reference pressure value of the crane of the second appearance type within a physical data real-time monitoring period, and obtaining a period reference pressure value; Step S237: Obtain the difference between the T1 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T1 pressure monitoring deviation, obtain the difference between the T2 monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the T2 pressure monitoring deviation, and so on, obtain the difference between the Tb monitoring pressure value and the periodic reference pressure value, and take the absolute value of the obtained difference to obtain the Tb pressure monitoring deviation; Step S238: Calculate the average of the pressure monitoring deviation from T1 to Tb to obtain the period average pressure monitoring deviation; Step S239: Calculate the periodic average pressure deviation and the pressure monitoring coefficient T1 to the pressure monitoring coefficient Tb to obtain the crane pressure periodic monitoring coefficient; Calculate the crane pipe pressure cycle monitoring coefficient.

6. The method for fully automatic crane status recognition based on visual image recognition according to claim 4 is characterized in that: The step S233 further includes the following specific steps: Step S2331: respectively obtaining the monitoring pressure value corresponding to each pressure monitoring sensor at the pressure monitoring point T1 to obtain a plurality of monitoring pressure values; Step S2332: Calculate the average of the obtained multiple monitoring pressure values ​​to obtain the T1 monitoring pressure value; Step S2333: performing variance calculation on the obtained multiple monitoring pressure values ​​to obtain T1 monitoring pressure variance; Step S2334: Calculate the T1 monitoring pressure value and the T1 monitoring pressure variance to obtain the pressure monitoring coefficient corresponding to the pressure monitoring sensor at the T1 pressure monitoring point, and name it T1 pressure monitoring coefficient; Calculate the T1 pressure monitoring coefficient.

7. The method for fully automatic crane status recognition based on visual image recognition according to claim 4 is characterized in that: The step S24 further includes the following specific steps: Step S241: multiple temperature monitoring sensors are arranged inside the crane tube of the second appearance type; Step S242: marking a number of temperature monitoring time points with equal time intervals in the physical data real-time monitoring cycle, and naming the marked temperature monitoring time points W1 temperature monitoring time point to Wc temperature monitoring time point in chronological order; Step S243: Obtain W1 monitoring temperature value and W1 temperature monitoring coefficient; Step S244: respectively acquiring the temperature monitoring coefficients corresponding to the temperature monitoring time point W2 to the temperature monitoring time point Wc, and obtaining the temperature monitoring coefficients W2 to Wc; Step S245: respectively acquiring the monitoring temperature values ​​corresponding to the temperature monitoring time point W2 to the temperature monitoring time point Wc, and obtaining the monitoring temperature values ​​W2 to Wc; Step S246: obtaining a reference temperature value of the crane of the second appearance type within a physical data real-time monitoring period to obtain a period reference temperature value; Step S247: Obtain the difference between the W1 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W1 temperature monitoring deviation, obtain the difference between the W2 monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the W2 temperature monitoring deviation, and so on, obtain the difference between the Wc monitoring temperature value and the periodic reference temperature value, and take the absolute value of the obtained difference to obtain the Wc temperature monitoring deviation; Step S248: Calculate the average of the W1 temperature monitoring deviation to the Wc temperature monitoring deviation to obtain the period average temperature monitoring deviation; Step S249: Calculate the periodic average temperature deviation and the W1 temperature monitoring coefficient to the Wc temperature monitoring coefficient to obtain the crane tube temperature periodic monitoring coefficient; Calculate the crane pipe temperature cycle monitoring coefficient.

8. The method for fully automatic crane status recognition based on visual image recognition according to claim 7 is characterized in that: The step S243 further includes the following specific steps: Step S2431: respectively obtain the monitoring temperature value corresponding to each temperature monitoring sensor at the W1 temperature monitoring point to obtain multiple monitoring temperature values, average the obtained multiple monitoring temperature values ​​to obtain the W1 monitoring temperature value, and perform variance calculation on the obtained multiple monitoring temperature values ​​to obtain the W1 monitoring temperature variance; Step S2432: Calculate the W1 monitoring temperature value and the W1 monitoring temperature variance to obtain the temperature monitoring coefficient corresponding to the temperature monitoring sensor at the W1 temperature monitoring point, and name it W1 temperature monitoring coefficient; Calculate the W1 temperature monitoring coefficient.

9. The method for fully automatic crane status recognition based on visual image recognition according to claim 1 is characterized in that: The step S3 further includes the following specific steps: Step S31: acquiring the crane appearance type classification data, and acquiring the first appearance type crane and the second appearance type crane according to the crane appearance type classification data; Step S32: if the automated crane is a crane of the first appearance type, determining that the automated crane is in an abnormal working state, and issuing a crane working state warning; Step S33: if the automated crane is of the second appearance type, the working state of the automated crane is determined according to the physical monitoring data of the crane, and an abnormal state warning is issued to the automated crane according to the determination result; The step S33 further includes the following specific steps: Step S331: obtaining the crane pipe physical monitoring data, and obtaining the crane pipe temperature cycle monitoring coefficient and the crane pipe pressure cycle monitoring coefficient respectively according to the crane pipe physical monitoring data; Step S332: obtaining a crane state judgment coefficient by calculating a crane temperature periodic monitoring coefficient and a crane pressure periodic monitoring coefficient; Calculate the crane pipe status judgment coefficient; Step S333: respectively obtaining a threshold value of a periodic monitoring coefficient of a temperature of a crane and a threshold value of a periodic monitoring coefficient of a pressure of a crane; Step S334: The crane state judgment coefficient threshold is obtained by calculating the crane temperature cycle monitoring coefficient threshold and the crane pressure cycle monitoring coefficient threshold; Step S335: if the crane state judgment coefficient is greater than or equal to the crane state judgment coefficient threshold, it is determined that the automated crane working state is abnormal, and a crane working state warning is issued; Step S336: If the crane state judgment coefficient is less than the crane state judgment coefficient threshold, it is determined that the automated crane working state is normal, and no crane working state warning is issued.

10. A fully automatic crane status recognition system based on visual image recognition, applicable to a fully automatic crane status recognition method based on visual image recognition as claimed in any one of claims 1 to 9, characterized in that: The state recognition system comprises: Visual image module: used to obtain appearance images of crane pipes at multiple different positions, obtain multiple crane pipe appearance images, create an appearance image recognition model to recognize multiple crane pipe appearance images, and classify the fully automatic crane pipes into first appearance type crane pipes and second appearance type crane pipes according to the recognition results, and obtain crane pipe appearance type classification data; Physical data module: used to perform periodic pressure monitoring and periodic temperature monitoring on the second appearance type crane in working state, and obtain the crane pressure periodic monitoring coefficient and the crane temperature periodic monitoring coefficient respectively by analyzing the monitoring results, and obtain the crane physical monitoring data; Status recognition module: used to identify the working state of the fully automatic crane according to the crane appearance type classification data and the crane physical monitoring data, and issue working state warnings based on the recognition results.

Images