Image-based temperature identification method and device, terminal equipment and storage medium

Abstract

The invention discloses an image-based temperature identification method and device, terminal equipment and a storage medium, and belongs to the field of image processing, and the method comprises the steps: obtaining a to-be-detected image of a to-be-detected sample; extracting chromaticity information of the to-be-detected image to obtain a chromaticity analysis matrix; inputting the chromaticity analysis matrix into a temperature identification model to generate the temperature of the to-be-tested sample; wherein the training of the temperature identification model specifically comprises the steps of obtaining historical image samples marked with temperature labels, forming a plurality of image libraries by the historical image samples of all temperature intervals according to a preset image quantity proportion, and training a machine learning model based on the image libraries, and obtaining a temperature identification initial model and a mean absolute error value corresponding to each image library, and selecting the temperature identification initial model with the minimum mean absolute error value as a temperature identification model. Therefore, by implementing the temperature identification method and device, the problem of low efficiency of temperature identification through an infrared camera in the prior art can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an image-based temperature identification method and device, a terminal device and a storage medium. BACKGROUND

[0002] In the operation process of a substation, accurate identification of the temperature rise fault of fittings is crucial, and is directly related to the stability and safety of the power system. Accurate identification of the temperature rise fault of fittings can timely discover potential safety hazards and avoid equipment failure and power outage accidents caused by overheating of fittings. Infrared temperature measurement is a commonly used temperature measurement method at present. However, infrared cameras are expensive and inconvenient to carry. In addition, due to the low pixel of the infrared camera, visible light images need to be combined for auxiliary positioning, so that temperature identification can be performed through the infrared camera. This leads to the problem of low efficiency of temperature identification through the infrared camera. SUMMARY

[0003] The present application provides an image-based temperature identification method and device, a terminal device and a storage medium, which can solve the problem of low efficiency of temperature identification through the infrared camera in the prior art.

[0004] The image-based temperature identification method provided by the present application comprises:

[0005] Obtaining a to-be-measured image of a to-be-measured sample;

[0006] Extracting chrominance information from the to-be-measured image to obtain a chrominance analysis matrix;

[0007] Inputting the chrominance analysis matrix into a temperature identification model to generate the temperature of the to-be-measured sample; wherein the training of the temperature identification model is specifically:

[0008] Obtaining historical image samples marked with temperature labels, grouping the historical image samples according to the image quantity proportion of each temperature interval to form an image library, and dividing the image library into a training set and a test set according to a predetermined proportion; wherein the temperature interval includes a normal temperature interval and a high temperature interval;

[0009] Training a machine learning model with the training set to obtain a temperature identification initial model, and calculating the mean absolute error value of the temperature identification initial model with the test set. When the mean absolute error value converges, the temperature identification initial model with the smallest mean absolute error value is selected as the temperature identification model.

[0010] Further, the obtaining of the historical image samples marked with temperature labels comprises:

[0011] Obtaining image samples corresponding to the metal sheet under heating;

[0012] judging a shooting direction of each image sample;

[0013] If the shooting direction is a front shooting direction, the current image sample is a front image sample, a measurement temperature of the current front image sample is measured, and the front image sample is labeled based on the measurement temperature to obtain a front image sample labeled with a temperature label;

[0014] If the shooting direction is a left-biased shooting direction, the current image sample is a left-biased image sample, a measurement temperature of the current left-biased image sample is measured, and the left-biased image sample is labeled based on the measurement temperature to obtain a left-biased image sample labeled with a temperature label;

[0015] All front image samples labeled with a temperature label and all left-biased image samples labeled with a temperature label are summarized to obtain historical image samples labeled with a temperature label.

[0016] Further, the image quantity ratio includes a first image quantity ratio and a second image quantity ratio; and the grouping of the historical image samples into the image library according to the image quantity ratio of each temperature interval includes:

[0017] The historical image samples are divided into a normal-temperature image set and a high-temperature image set based on the temperature interval and the sample type to which the temperature label of the historical image sample belongs;

[0018] Metal type judgment is performed on the metal sheet corresponding to the historical image sample;

[0019] If the metal type is a first metal type, a first high-temperature sample is selected from the high-temperature image set, and the first high-temperature sample and the normal-temperature image set are grouped into the image library; wherein the number of the first high-temperature sample accounts for a first image quantity ratio of the total number of images in the normal-temperature image set;

[0020] If the metal type is a second metal type, a second high-temperature sample is selected from the high-temperature image set, and the second high-temperature sample and the normal-temperature image set are grouped into the image library; wherein the number of the second high-temperature sample accounts for a second image quantity ratio of the total number of images in the normal-temperature image set.

[0021] Further, the grouping of the historical image samples into the image library according to the image quantity ratio of each temperature interval includes:

[0022] The temperature interval and the sample type to which the temperature label of each historical image sample belongs are judged;

[0023] If the sample type is a front shooting direction, the current historical image sample belongs to the normal-temperature image set;

[0024] If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the first temperature interval, the current historical image sample belongs to the normal temperature image set.

[0025] If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the second temperature interval, the current historical image sample belongs to the high-temperature image set.

[0026] Further, the chroma analysis matrix comprises: a gray matrix corresponding to the three primary color channels of the to-be-tested image; wherein the primary color channels comprise: a red channel, a green channel and a blue channel; and the gray matrix comprises: a mean value, a standard deviation, a kurtosis, a skewness, a mode, a median, a peak value of the gray matrix, an upper alpha quantile point and a lower alpha quantile point.

[0027] Further, the training of the machine learning model with the training set to obtain a temperature recognition initial model, and the calculation of the mean absolute error value of the temperature recognition initial model with the test set, when the mean absolute error value converges, selecting the temperature recognition initial model with the minimum mean absolute error value as the temperature recognition model, comprises:

[0028] The model training operation is repeatedly executed, and when the mean error value converges, the model training operation is stopped, and the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model; wherein the model training operation is specifically:

[0029] The current machine learning model is trained based on the training set to obtain a current temperature recognition initial model, the temperature of the test set is predicted based on the temperature recognition initial model to obtain a predicted temperature, the mean absolute error value of the temperature recognition initial model is calculated based on the predicted temperature and the temperature label corresponding to the test set, whether the mean error value converges is determined according to the mean absolute error value, if yes, the model training operation is stopped; if no, the model parameters of the machine learning model are adjusted, and the machine learning model after the model parameter adjustment is taken as the machine learning model of the next model training operation.

[0030] Further, the calculation of the mean absolute error value of the temperature recognition initial model based on the predicted temperature and the temperature label corresponding to the test set comprises:

[0031] The predicted temperature corresponding to the temperature recognition initial model and the temperature label corresponding to the test set are substituted into a preset mean absolute error calculation formula to calculate the mean absolute error of the temperature recognition initial model; wherein the mean absolute error calculation formula is specifically:

[0032]

[0033] In the formula, MAE is the mean absolute error, y represents the predicted temperature of the test set by the temperature recognition initial model, and y represents the temperature label corresponding to the test set.i temperature value of a temperature label corresponding to a test set, n is the number of samples in the test set.

[0034] Another embodiment of the present application also provides an image-based temperature identification device, comprising a data acquisition module, a data extraction module and a result generation module.

[0035] The data acquisition module is configured to acquire a to-be-tested image of a to-be-tested sample.

[0036] The data extraction module is configured to extract chrominance information of the to-be-tested image to obtain a chrominance analysis matrix.

[0037] The result generation module is configured to input the chrominance analysis matrix into a temperature identification model to generate a temperature of the to-be-tested sample.

[0038] The historical image samples marked with temperature labels are acquired, the historical image samples are grouped into an image library according to the image quantity proportion of each temperature interval, and the image library is divided into a training set and a test set according to a preset proportion; the temperature interval includes a normal temperature interval and a high temperature interval.

[0039] The machine learning model is trained by using the training set to obtain a temperature identification initial model, the average absolute error value of the temperature identification initial model is calculated by using the test set, and when the average absolute error value converges, the temperature identification initial model with the minimum average absolute error value is selected as the temperature identification model.

[0040] Another embodiment of the present application also provides a terminal device, comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, when the computer program is executed by the processor, the steps of the image-based temperature identification method provided by the present application are implemented.

[0041] Another embodiment of the present application also provides a computer readable storage medium item, comprising a stored computer program, when the computer program is running, the device where the computer readable storage medium is located is controlled to execute the steps of the image-based temperature identification method provided by the present application.

[0042] By implementing the present application, the following beneficial effects are achieved:

[0043] The present application can obtain more image features of the to-be-tested image by acquiring the to-be-tested image of the to-be-tested sample, extracting the chrominance information of the to-be-tested image, and obtaining the chrominance analysis matrix, and by inputting the chrominance analysis matrix into the temperature recognition model, the temperature recognition model can deeply learn the image features of the to-be-tested image, and then output the temperature of the to-be-tested sample based on the chrominance analysis matrix. Meanwhile, when training the temperature recognition model, the historical image samples marked with temperature labels are respectively formed into image libraries according to the image quantity proportion of each temperature interval, and the training set and the test set of the image library are divided, so as to train the machine learning model based on the training set, calculate the mean absolute error of the temperature recognition initial model obtained by training through the test set, and select the temperature recognition initial model with the minimum mean absolute error as the temperature recognition model when the mean absolute error converges, thereby completing the training of the temperature recognition model. The present application realizes temperature recognition based on images, without the assistance of infrared light for temperature recognition, and greatly improves the efficiency of temperature recognition. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0045] Figure 1 is a flowchart of the temperature recognition method based on images provided by an embodiment of the present application;

[0046] Figure 2 is a structural schematic diagram of the temperature recognition device based on images provided by an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of the three-primary-color gray scale probability distribution provided by an embodiment of the present application;

[0048] Figure 4 is a schematic diagram of the type of database provided by an embodiment of the present application;

[0049] Figure 5 is a model error schematic diagram of the material copper provided by an embodiment of the present application;

[0050] Figure 6 is a model error schematic diagram of the material iron provided by an embodiment of the present application;

[0051] Figure 7 is a model error schematic diagram of the material aluminum provided by an embodiment of the present application;

[0052] Figure 8is a recognition error schematic diagram of copper under different proportions of U-bx library (rare sample update library) in scheme one and scheme two provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] For the purposes of the present application, the technical solutions and advantages are more clearly, the technical solutions in the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0055] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise explicitly and specifically limited.

[0056] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0058] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0059] In the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the technical terms "mounting", "connection", "connecting", "fixing" and the like should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integral; can be mechanical connection, can also be electrical connection; can be direct connection, can also be indirect connection through an intermediate medium, can be internal communication of two elements or interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0060] Referring to Figure 1 To solve the problem of low efficiency of temperature identification by an infrared camera in the prior art, an embodiment of the present application provides a temperature identification method based on images, comprising:

[0061] 101, obtaining a to-be-tested image of a to-be-tested sample.

[0062] 102, extracting chrominance information of the to-be-tested image to obtain a chrominance analysis matrix.

[0063] Further, the chrominance analysis matrix comprises: a gray matrix corresponding to three primary color channels of the to-be-tested image; wherein the primary color channels comprise: a red channel, a green channel and a blue channel; and the gray matrix comprises: a mean value, a standard deviation, a kurtosis, a skewness, a mode, a median, a peak value of the gray matrix, an upper alpha quantile point and a lower alpha quantile point.

[0064] In a specific embodiment, referring to Figure 3 The color of each pixel of a picture can be mixed by three primary colors R (red), G (green) and B (blue) of different gray levels (0-255, a total of 256 gray levels), and the higher the gray level of a color, the higher its color brightness. The gray probability refers to the proportion of the primary color of the gray level to all pixel points of the picture, and the formula is as follows:

[0065]

[0066] Wherein, NumL(i) represents the number of pixel points of the corresponding primary color at the gray level.

[0067] The gray probability curve of the picture of the gold fitting at different temperatures will change greatly, and we use 9 characteristic quantities under each primary color channel to indicate the change of the gray probability curve characteristics, and the mean value, standard deviation, kurtosis, skewness, mode, median and peak value of the gray matrix, upper alpha quantile point and lower alpha quantile point of the corresponding primary color gray matrix are taken. There are 9*3=27-dimensional chrominance analysis matrices.

[0068] 103. Input the chromaticity resolution matrix into the temperature recognition model to generate the temperature of the sample to be tested; wherein, the training of the temperature recognition model specifically involves:

[0069] Acquire historical image samples labeled with temperature tags, and form an image library by the proportion of images in each temperature range. Divide the image library into training and testing sets according to a preset ratio. The temperature ranges include: normal temperature range and high temperature range.

[0070] The machine learning model is trained using the training set to obtain an initial temperature recognition model. The mean absolute error (MAE) of the initial temperature recognition model is then calculated using the test set. When the MAE converges, the initial temperature recognition model with the smallest MAE is selected as the temperature recognition model.

[0071] In one specific embodiment, the machine learning algorithms used are k-nearest neighbor (kNN) based on Euclidean distance, decision tree (DT), gradient boosting regression tree (GBRT) based on boosting algorithm, and random forest regression (RFR) based on bagging algorithm.

[0072] Further, acquiring historical image samples labeled with temperature tags includes:

[0073] Acquire image samples corresponding to the metal sheet being heated;

[0074] Determine the shooting direction for each image sample;

[0075] If the shooting direction is facing the camera, the current image sample is the facing image sample. The temperature of the current facing image sample is measured, and the facing image sample is labeled based on the measured temperature to obtain the facing image sample with the temperature label.

[0076] If the shooting direction is left-leaning, the current image sample is a left-leaning image sample. The temperature of the current left-leaning image sample is measured, and the left-leaning image sample is labeled based on the measured temperature to obtain a left-leaning image sample labeled with temperature.

[0077] By summing all the front-facing image samples and all the left-leaning image samples labeled with temperature, we can obtain historical image samples labeled with temperature.

[0078] In a specific embodiment, the temperature of the metal sheet is heated by the heating device, the temperature adjustment range is 10-100°C, and the adjustment accuracy is 0.1°C; the Hikvision DS-2CD3T86WDV2-I5S camera is used as the video image acquisition device; the metal temperature is measured in real time by the R-81531AK thermocouple and the Beckers BK8802U dual-channel thermometer; and the environmental light source is four 60w incandescent lamps.

[0079] The images of the metal sheet are extracted at fixed time intervals from the video, and the image set corresponding to the metal is obtained. On this basis, the picture with a specified pixel size (65*80) and the real-time temperature measured by the corresponding temperature measuring device are obtained as the picture label, and the image library is constituted.

[0080] The materials of the fittings are clean and smooth copper sheets, clean and smooth iron sheets, and clean and smooth aluminum sheets. For each material, an image set of 4800 pictures of the material from 30°C to 100°C in two different shooting directions is established, including a straight-on shooting library A library and a left-biased shooting library U library. Each database contains 4800 pictures, and a total of 6 databases are established.

[0081] In a specific embodiment, the on-site temperature measurement technology based on visible light digital images and machine learning technology generally first establishes a full-temperature-range database A (30-100°C) in the laboratory, studies image features, and trains a model with good temperature recognition accuracy. Then, a part of the image library is collected on site as an update database U, which is mixed into the full-temperature-range database A to obtain an actual training library F to train the model. There are many images in the normal temperature range on site, but there are fewer images in high temperature (fault temperature). The purpose of this embodiment is to find the minimum number x of rare sample updates or the proportion of the database U-bx. In the process of finding the lowest addition proportion of rare sample images and the optimal machine learning algorithm with high recognition accuracy, the number of the update database U-bx is adjusted to find the lowest addition proportion and the optimal algorithm. Specifically, the sample image library added is input into the temperature recognition model, and the model is trained through machine learning. Each machine learning training uses four different machine learning algorithms to train the temperature recognition model. The trained model is used to recognize the temperature of the fittings on site, and the temperature recognition results of different addition proportions and machine learning algorithms are obtained.

[0082] Referring to Figure 4 , the type of the image library is:

[0083] 1) A library: a shooting library with the lens directly facing the object (30-100°C, 4800 pictures) as a full-temperature-range database established in the laboratory, used for basic training to find some suitable image features and algorithms;

[0084] 2) U library: left object shooting library of lens (30-100℃ 4800 pictures) as an update database of the full temperature range in the field, used for sample updating and improving model recognition accuracy before use in different scenes in the field.

[0085] wherein

[0086] a) U-a library (1370 pictures): represents an update database of the normal temperature interval 30-50℃ that can be collected in the field.

[0087] b) U-b library (3430 pictures): represents a rare sample update database of the high temperature interval 51-100℃ that is not easy to collect in the field (much smaller than the sample number during training).

[0088] c) U-bx library (x pictures) 51-100℃: x pictures from the U-b library. The purpose of the present application is to find the minimum sample number, i.e. the minimum value of x, under the premise of meeting the recognition accuracy.

[0089] 3) F library: basic training library,

F

A

U-a

[0090] 4) Actual training library:

F

U-bx

U-bx

F

[0091] Further, the image quantity proportion includes a first image quantity proportion and a second image quantity proportion; and the historical image samples are grouped into the image library according to the image quantity proportion of each temperature interval, including:

[0092] The historical image samples are divided into a normal temperature image set and a high temperature image set based on the temperature interval and sample type of the temperature label of the historical image samples;

[0093] Metal type judgment is performed on the metal sheet corresponding to the historical image samples;

[0094] If the metal type is a first metal type, a first high temperature sample is selected from the high temperature image set, and the first high temperature sample and the normal temperature image set form an image library; wherein the proportion of the number of the first high temperature sample to the number of all images in the normal temperature image set is a first image quantity proportion;

[0095] If the metal type is a second metal type, a second high temperature sample is selected from the high temperature image set, and the second high temperature sample and the normal temperature image set form an image library; wherein the proportion of the number of the second high temperature sample to the number of all images in the normal temperature image set is a second image quantity proportion.

[0096] The first image quantity ratio is a minimum ratio value of the first metal category, and the second image quantity ratio is a minimum ratio value of the second metal category.

[0097] In a specific embodiment, the iron and copper materials (i.e., the first metal category) need to add more than 2% (i.e., the first image quantity ratio) of the number of first high-temperature samples in the original base library to make the model prediction error of rare samples drop below 3°C, meeting the needs of on-site identification; aluminum (i.e., the second metal category) needs to add more than 10% (i.e., the second image quantity ratio) of the number of second high-temperature samples in the original base library to achieve the same effect.

[0098] Due to the small sample size of high-temperature samples, in order to ensure the recognition accuracy of high temperature, the first image quantity ratio and the second image quantity ratio are studied by comparing the temperature recognition effects of multiple different data and different balance degrees.

[0099] Three materials and two different data imbalance conditions are mainly studied, and a total of six machine learning training groups are performed. Each machine learning training uses four different machine learning algorithms (kNN, DT, GBRT, and RFR), and two division schemes (see Table 1) are compared. The data imbalance condition and machine learning training details are as follows:

[0100] The A library of the straight-on shooting library and the U-a library of the left-biased shooting library are merged to form the base library F library; from the left-biased shooting library U library, the medium-high temperature and high temperature pictures in the rare sample are extracted as the rare sample update library U-b. A certain number of samples are randomly extracted from the U-b library each time, and gradually added to the F library. The number of samples extracted from the U-b library each time is different, resulting in dynamic changes in the imbalance degree of the on-site samples, and the F library after each update is trained using four machine learning algorithms: kNN, DT, GBRT, and RFR. The proportion of U library samples is gradually increased, and when 0-10% and 90-100% of the U library are added, the proportion of U library pictures added each time differs by 1% of the total number of U library pictures, to finely capture the performance changes of the data imbalance problem. In other cases, the proportion of U library pictures added each time differs by 10%, and a total of 29 different data imbalance degrees are trained. The mean absolute error MAE is used as the model measurement index: when the temperature error (MAE) of the rare update sample detection is below 3°C, it can reflect the technical features of solving the data imbalance problem.

[0101] Table 1

[0102]

[0103] Scheme 1, the failure temperature range is set large, i.e. U-bx is 51-100℃; h =

U-bx

F

[0104] Scheme 2, the failure temperature range is set small, i.e. U-bx is 81-100℃; h =

U-bx

F

[0105] To evaluate the accuracy and error of the machine learning model for temperature prediction, this paper mainly uses the mean absolute error (MAE):

[0106]

[0107] As a measure, the expression in the picture model represents the predicted temperature of the picture model, y i represents the actual temperature of the picture.

[0108] MAE reflects the average of the absolute value of the error between the predicted temperature and the actual label temperature, that is, the smaller the MAE, the closer to 0, the higher the accuracy of the model.

[0109] The training image library is divided by k-fold cross-validation (k = 10): the training image library is evenly divided into 10 groups, and each time the model is trained using 9 groups as the training set, and the remaining 1 group is used as the validation set to evaluate the model error. Repeat the above process 10 times to ensure that each group of data is used as a validation set once, and finally select the model with the smallest error as the optimal model. The test set is extracted from the remaining 10% of the update library (U library), ensuring that the test set is completely independent of the training set. By changing the proportion of rare samples (high-temperature pictures) in the training set, the curve of model error (MAE) with the proportion of rare samples is drawn, as follows: see Figure 5 , the material is copper, the recognition error of scheme one U-bx library (rare sample update library) with different proportions, the proportion h = x / (4800+1370); see Figure 6 , the material is iron, the recognition error of scheme one U-bx library (rare sample update library) with different proportions, the proportion h = x / (4800+1370); see Figure 7 , the material is aluminum, the recognition error of scheme one U-bx library (rare sample update library) with different proportions, the proportion h = x / (4800+1370). Compared with DT and kNN algorithms, RFR and GBRT algorithms perform well with smaller errors.

[0110] Based on the above, the fault temperature range is set small, that is, when the rare sample is a high temperature working condition picture on site (81-100℃), and the data imbalance of the above data structure, the temperature error of the four algorithms also decreases with the decrease of the data imbalance degree, and the error of the three different gold materials can also be below 3℃, which can accurately detect the picture temperature under high temperature working condition on site. In addition, see Figure 8 Different fault temperature setting ranges have little effect on the above conclusion.

[0111] Further, the temperature interval and sample type to which the temperature label of the historical image sample belongs are used to divide the historical image sample into a normal temperature image set and a high temperature image set, including:

[0112] The temperature interval and sample type to which the temperature label of each historical image sample belongs are determined;

[0113] If the sample type is a normal shooting direction, the current historical image sample belongs to the normal temperature image set;

[0114] If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the first temperature interval, the current historical image sample belongs to the normal temperature image set;

[0115] If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the second temperature interval, the current historical image sample belongs to the high temperature image set.

[0116] Further, the machine learning model is trained with the training set to obtain a temperature recognition initial model, and the average absolute error value of the temperature recognition initial model is calculated with the test set. When the average absolute error value converges, the temperature recognition initial model with the smallest average absolute error value is selected as the temperature recognition model, including:

[0117] The model training operation is repeatedly performed, and when the average error value converges, the model training operation is stopped, and the temperature recognition initial model with the smallest average absolute error value is selected as the temperature recognition model; wherein the model training operation is specifically:

[0118] The current machine learning model is trained based on the training set to obtain a current temperature recognition initial model, the temperature of the test set is predicted based on the temperature recognition initial model to obtain a predicted temperature, the average absolute error value of the temperature recognition initial model is calculated based on the predicted temperature and the temperature label corresponding to the test set, and whether the average error value converges is determined according to the average absolute error value. If yes, stop performing the model training operation; if not, adjust the model parameters of the machine learning model, and use the machine learning model after adjusting the model parameters as the machine learning model of the next model training operation.

[0119] Further, the average absolute error value of the temperature identification initial model is calculated based on the predicted temperature and the temperature label corresponding to the test set, and the average absolute error value of the temperature identification initial model is calculated based on the predicted temperature and the temperature label corresponding to the test set.

[0120] The predicted temperature corresponding to the temperature identification initial model and the temperature label corresponding to the test set are substituted into the preset average absolute error calculation formula to calculate the average absolute error of the temperature identification initial model.

[0121]

[0122] In the formula, MAE is the average absolute error, y represents the predicted temperature of the temperature identification initial model for the test set, and y i represents the temperature value of the temperature label corresponding to the test set, and n is the number of samples in the test set.

[0123] As Figure 2 shown, on the basis of the above method item embodiment, corresponding device item embodiments are provided;

[0124] An embodiment of the present application provides a temperature identification device based on images, comprising: a data acquisition module 201, a data extraction module 202 and a result generation module 203.

[0125] The data acquisition module is used to acquire the to-be-tested image of the to-be-tested sample.

[0126] The data extraction module is used to extract the chrominance information of the to-be-tested image to obtain a chrominance analysis matrix.

[0127] The result generation module is used to input the chrominance analysis matrix into a temperature identification model to generate the temperature of the to-be-tested sample; wherein the training of the temperature identification model is specifically:

[0128] The historical image samples marked with temperature labels are acquired, the historical image samples are grouped into an image library according to the image quantity proportion of each temperature interval, and the image library is divided into a training set and a test set according to a preset proportion; wherein the temperature interval includes a normal temperature interval and a high temperature interval.

[0129] The machine learning model is trained with the training set to obtain a temperature identification initial model, and the average absolute error value of the temperature identification initial model is calculated with the test set; when the average absolute error value converges, the temperature identification initial model with the minimum average absolute error value is selected as the temperature identification model.

[0130] It can be understood that the above device item embodiment corresponds to the method item embodiment of the present application, and can realize the temperature identification method based on images provided by any one of the above method item embodiments.

[0131] It should be noted that the apparatus embodiments described above are merely illustrative, and part or all of the modules thereof can be selected to achieve the purposes of the embodiments of the present application according to actual needs. In addition, in the apparatus embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0132] On the basis of the above-mentioned embodiments of the image-based temperature identification method, another embodiment of the present application provides a terminal device, which comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the image-based temperature identification method of any one of the embodiments of the present application is realized.

[0133] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device.

[0134] The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing devices. The terminal device can include, but is not limited to, a processor and a memory.

[0135] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and connects all parts of the terminal device through various interfaces and lines.

[0136] On the basis of the above method embodiment, another embodiment of the present application provides a computer readable storage medium, comprising a stored computer program, wherein the computer readable storage medium controls the device to execute the image-based temperature identification method of any one of the above method embodiments of the present application when the computer program is running.

[0137] The modules / units integrated in the device / terminal equipment, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0138] The above is the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered to be within the scope of protection of the present application.

Claims

1. An image-based temperature identification method, characterized by, The method comprises the following steps: acquiring a to-be-tested image of a to-be-tested sample; extracting chrominance information of the to-be-tested image to obtain a chrominance analysis matrix; inputting the chrominance analysis matrix into a temperature identification model to generate a temperature of the to-be-tested sample; wherein the training of the temperature identification model comprises the following steps: acquiring historical image samples marked with temperature labels, grouping the historical image samples into an image library according to the proportion of the number of images in each temperature interval, and dividing the image library into a training set and a test set according to a preset proportion; wherein the temperature intervals include a normal temperature interval and a high temperature interval; training a machine learning model with the training set to obtain a temperature identification initial model, and calculating the mean absolute error value of the temperature identification initial model with the test set; when the mean absolute error value converges, the temperature identification initial model with the minimum mean absolute error value is selected as the temperature identification model.

2. The image-based temperature identification method of claim 1, wherein, The method of acquiring historical image samples marked with temperature labels comprises the following steps: acquiring image samples corresponding to a metal sheet under heating; judging the shooting direction of each image sample; if the shooting direction is a direct shooting direction, the current image sample is a direct image sample, the measurement temperature of the current direct image sample is measured, and the direct image sample is labeled based on the measurement temperature to obtain a direct image sample marked with a temperature label; if the shooting direction is a left-biased shooting direction, the current image sample is a left-biased image sample, the measurement temperature of the current left-biased image sample is measured, and the left-biased image sample is labeled based on the measurement temperature to obtain a left-biased image sample marked with a temperature label; all direct image samples marked with temperature labels and all left-biased image samples marked with temperature labels are collected to obtain historical image samples marked with temperature labels.

3. The image-based temperature identification method of claim 2, wherein, The image number proportion includes a first image number proportion and a second image number proportion; the grouping of the historical image samples into an image library according to the proportion of the number of images in each temperature interval comprises the following steps: based on the temperature interval and the sample type to which the temperature label of the historical image sample belongs, the historical image samples are divided into a normal temperature image set and a high temperature image set; judging the metal type of the metal sheet corresponding to the historical image sample; if the metal type is a first metal type, a first high temperature sample is selected from the high temperature image set, and the first high temperature sample and the normal temperature image set form an image library; wherein the proportion of the number of the first high temperature sample to the number of all images in the normal temperature image set is the first image number proportion; if the metal type is a second metal type, a second high temperature sample is selected from the high temperature image set, and the second high temperature sample and the normal temperature image set form an image library; wherein the proportion of the number of the second high temperature sample to the number of all images in the normal temperature image set is the second image number proportion.

4. The image-based temperature identification method of claim 3, wherein, The grouping of the historical image samples into an image library according to the proportion of the number of images in each temperature interval comprises the following steps: judging the temperature interval and the sample type to which the temperature label of each historical image sample belongs; if the sample type is a direct shooting direction, the current historical image sample belongs to the normal temperature image set; If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the first temperature interval, the current historical image sample belongs to the normal temperature image set. If the sample type is a left-biased shooting direction and the temperature interval to which the temperature label belongs is the second temperature interval, the current historical image sample belongs to the high-temperature image set.

5. The image-based temperature identification method of claim 4, wherein, The chroma analysis matrix comprises: a gray matrix corresponding to three primary color channels of the to-be-tested image; wherein the primary color channels comprise: a red channel, a green channel and a blue channel; and the gray matrix comprises: a mean value, a standard deviation, a kurtosis, a skewness, a mode, a median, a peak value of the gray matrix, an upper alpha quantile point and a lower alpha quantile point.

6. The image-based temperature identification method of claim 5, wherein, The training set is used to train the machine learning model to obtain a temperature recognition initial model, and the test set is used to calculate the mean absolute error value of the temperature recognition initial model. When the mean absolute error value converges, the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model, comprising: The model training operation is repeatedly executed, and when the mean error value converges, the model training operation is stopped, and the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model; wherein the model training operation is specifically: The training set is used to train the machine learning model to obtain a temperature recognition initial model, and the test set is used to calculate the mean absolute error value of the temperature recognition initial model. When the mean absolute error value converges, the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model, comprising:

7. The image-based temperature identification method of claim 6, wherein, The training set is used to train the machine learning model to obtain a temperature recognition initial model, and the test set is used to calculate the mean absolute error value of the temperature recognition initial model. When the mean absolute error value converges, the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model, comprising: The training set is used to train the machine learning model to obtain a temperature recognition initial model, and the test set is used to calculate the mean absolute error value of the temperature recognition initial model. When the mean absolute error value converges, the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model, comprising: In the formula, MAE is the mean absolute error, represents the predicted temperature of the test set by the temperature recognition initial model, y i represents the temperature value of the temperature label corresponding to the test set, and n is the number of samples in the test set.

8. An image-based temperature identification device, characterized by, The training set is used to train the machine learning model to obtain a temperature recognition initial model, and the test set is used to calculate the mean absolute error value of the temperature recognition initial model. When the mean absolute error value converges, the temperature recognition initial model with the minimum mean absolute error value is selected as the temperature recognition model, comprising: It comprises: A data acquisition module, a data extraction module and a result generation module; The data acquisition module is used to acquire the to-be-tested image of the to-be-tested sample; The data extraction module is used to extract the chroma information of the to-be-tested image to obtain a chroma analysis matrix; The result generation module is used to input the chroma analysis matrix into the temperature recognition model to generate the temperature of the to-be-tested sample; wherein the training of the temperature recognition model is specifically: The historical image samples marked with temperature labels are acquired, the historical image samples are grouped into an image library according to the image quantity proportion of each temperature interval, and the image library is divided into a training set and a test set according to a preset proportion; wherein the temperature intervals comprise: a normal temperature interval and a high temperature interval; The machine learning model is trained by using the training set to obtain a temperature identification initial model, and the average absolute error value of the temperature identification initial model is calculated by using the test set; when the average absolute error value converges, the temperature identification initial model with the minimum average absolute error value is selected as the temperature identification model.

9. A terminal device, comprising: The image-based temperature identification method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the computer program is executed by the processor, the image-based temperature identification method according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that, The image-based temperature identification method comprises: A stored computer program, wherein when the computer program is running, the device where the computer readable storage medium is located is controlled to execute the image-based temperature identification method according to any one of claims 1-7.

Images