Device heating detection method and system based on HSV color space image fusion

Abstract

The invention discloses a device heating detection method and system based on HSV color space image fusion, and relates to the field of automobile manufacturing, and the method comprises the steps: converting the infrared image and visible light image data of a to-be-heated device into an HSV color space, and carrying out the channel separation; performing weighted fusion on the brightness channels of the infrared and visible light images, converting the fused HSV color space into an RGB image, and performing bilateral filtering on the obtained RGB image to obtain a fused image; and calculating the fused image based on a temperature identification algorithm of the target detection model to obtain real-time temperature information of the device to be heated. According to the invention, the detection efficiency is effectively improved, the image temperature can be accurately identified, and the temperature of the rearview mirror and the rear windshield of the automobile can be accurately sensed.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile intelligent manufacturing, and in particular to a device heating detection method and system based on HSV color space image fusion. Background Art

[0002] In rainy and snowy weather, the mirror surface of the rearview mirror and the glass surface of the rear windshield are easy to frost and fog, which will seriously affect the driver's field of vision and driving safety. Therefore, for driving safety, in rainy and snowy weather, the mirror surface of the rearview mirror and the glass surface of the rear windshield need to be heated. This is a common function on cars. When the car is manufactured, it is necessary to check whether this function is operating normally.

[0003] In vehicle factories, the heating function test of vehicle rearview mirrors and rear windshields is mainly carried out manually, mainly through visual or manual touch to confirm whether the heating function is normal. Relying only on subjective feeling, it can only roughly judge whether the temperature of the rearview mirrors and rear windshields has changed, and it is impossible to accurately perceive the temperature value of the glass. In addition, manual inspection is time-consuming and labor-intensive. Summary of the invention

[0004] In view of the defects existing in the prior art, the purpose of the present invention is to provide a device heating detection method and system based on HSV color space image fusion, which not only effectively improves the detection efficiency, but also can accurately identify the image temperature and accurately sense the temperature of the car rearview mirror and rear windshield.

[0005] In a first aspect, the present invention provides a device heating detection method based on HSV color space image fusion, comprising the following steps:

[0006] The infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation;

[0007] Perform weighted fusion of the brightness channels of infrared and visible light images;

[0008] Converting the fused HSV color space into an RGB image, and performing bilateral filtering on the RGB image to obtain a fused image;

[0009] A temperature recognition algorithm based on the target detection model is used to calculate the fused image and obtain the real-time temperature information of the device to be heated.

[0010] In one embodiment, the infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation, which specifically includes the following steps:

[0011] The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

[0012] In one embodiment, the weighted fusion of the brightness channels of the infrared and visible light images specifically includes the following steps:

[0013] Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively;

[0014] The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

[0015] In one implementation, the temperature recognition algorithm based on the target detection model is:

[0016] A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once;

[0017] The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

[0018] In one embodiment, after obtaining the real-time temperature information of the device to be heated, the following steps are also included:

[0019] On the monitoring system page, the real-time temperature information of the heated device is displayed, the vehicle identification code VIN code is recorded, and early warning prompts are issued for vehicles that do not meet the heating requirements.

[0020] In a second aspect, the present invention provides a device heating detection system based on HSV color space image fusion, comprising:

[0021] The HSV conversion unit is used to convert the infrared image and visible light image data of the device to be heated into the HSV color space for channel separation;

[0022] The weighted fusion unit is used to: perform weighted fusion on the brightness channels of infrared and visible light images;

[0023] RGB conversion unit, used to: convert the fused HSV color space into an RGB image;

[0024] A bilateral filtering unit, used to: perform bilateral filtering on the RGB image to obtain a fused image;

[0025] The temperature detection unit is used to: calculate the fused image based on the temperature recognition algorithm of the target detection model to obtain the real-time temperature information of the device to be heated.

[0026] In one embodiment, the HSV conversion unit is specifically used for:

[0027] The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

[0028] In one implementation, the weighted fusion unit is specifically used to:

[0029] Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively;

[0030] The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

[0031] In one implementation, the temperature detection unit uses a temperature recognition algorithm based on a target detection model:

[0032] A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once;

[0033] The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

[0034] In one embodiment, the system further comprises:

[0035] The display warning unit is used to: after obtaining the real-time temperature information of the device to be heated, display the real-time temperature information of the device to be heated, record the vehicle identification code VIN code, and issue a warning prompt to the vehicle that does not meet the heating requirements.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] (1) This application adopts infrared recognition to test the heating function of vehicle components, replacing the traditional detection method in which inspectors manually touch the rearview mirror and rear windshield. It not only effectively improves the detection efficiency, but also can accurately identify the image temperature and accurately sense the temperature of the car's rearview mirror and rear windshield.

[0038] (2) This application converts the RGB color space of infrared images and visible light into images of the HSV color space and then fuses them. The channels of the HSV model have strong independence and are closer to the human eye's perception of color, which improves the visual effect of image recognition. By weighted fusion of infrared images and visible light images under the HSV color space model, accurate values ​​of each channel of the HSV color space are obtained, which improves the segmentation accuracy of the target area of ​​image recognition and solves the problem of fuzzy target boundary texture segmentation in the case of single infrared image recognition.

[0039] (3) This application introduces a lightweight CBAM attention module through an improved yolov5s target detection model, and uses the α-CIoU loss function to replace the original prediction box boundary loss function, so as to effectively and accurately predict the temperature of the rearview mirror and the rear windshield, which is conducive to improving the problems of unclear boundaries and difficult network feature capture in infrared image recognition of heating devices of various models, and improving the detection performance.

[0040] (4) The monitoring system page corresponding to the design of this application obtains the real-time temperature information of the vehicle's heated components based on the image data, displays the real-time temperature information of the heated components on the monitoring system page, issues early warning prompts for vehicles that do not meet the heating requirements, and records the vehicle's VIN (Vehicle Identification Number) code, thereby realizing automatic recording and processing of thermal imaging image data and intuitively displaying infrared thermal imaging monitoring historical data, which is convenient for subsequent query and tracing of faulty vehicle information and analysis of fault problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is an overall flow chart of the heating device detection method based on HSV color space image fusion in an embodiment of the present invention.

[0043] Figure 2 The simplified flowchart of collecting and fusing infrared images and visible light images in an embodiment of the present invention is shown.

[0044] Figure 3 It is a detailed flow chart of fusing infrared images and visible light images in an embodiment of the present invention.

[0045] Figure 4 This is a simulation diagram for identifying the temperature value of the selected area in an embodiment of the present invention.

[0046] Figure 5 The following is a detailed flow chart of monitoring the temperature of a vehicle heating device in an embodiment of the present invention.

[0047] Figure 6 Schematic diagram of a heating device detection system based on HSV color space image fusion in an embodiment of the present invention.

[0048] Figure 7 It is an architecture diagram of a heating device detection system based on HSV color space image fusion in another embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0050] In one aspect, an embodiment of the present invention provides a device heating detection method based on HSV color space image fusion, comprising the following steps:

[0051] The infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation;

[0052] Perform weighted fusion of the brightness channels of infrared and visible light images;

[0053] Convert the fused HSV color space into an RGB image, perform bilateral filtering on the RGB image, and obtain the fused image;

[0054] A temperature recognition algorithm based on the target detection model is used to calculate the fused image and obtain the real-time temperature information of the device to be heated.

[0055] Furthermore, the infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation, which specifically includes the following steps:

[0056] The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

[0057] Furthermore, the brightness channels of the infrared and visible light images are weightedly fused, which specifically includes the following steps:

[0058] Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively;

[0059] The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

[0060] Furthermore, the temperature recognition algorithm based on the target detection model is:

[0061] A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once;

[0062] The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

[0063] Furthermore, after obtaining the real-time temperature information of the device to be heated, the following steps are also included:

[0064] On the monitoring system page, the real-time temperature information of the heated device is displayed, the vehicle identification code VIN code is recorded, and early warning prompts are issued for vehicles that do not meet the heating requirements.

[0065] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0066] See also Figure 1 As shown, the heating device detection method based on HSV color space image fusion in an embodiment of the present invention includes the following steps:

[0067] Step S1, connect the infrared image and visible light image acquisition devices, initialize them, and go to step S2;

[0068] Step S2, based on the DLL (Dynamic Link Library, also known as "application extension") file path, load the algorithm function and go to step S3;

[0069] Step S3, obtaining real-time infrared image and visible light image data of the device to be heated, and then proceeding to step S4;

[0070] Step S4, perform HSV color space image fusion processing on the acquired infrared image and visible light image data, and then go to step S5;

[0071] Step S5, displaying the processed image data on the monitoring system page, obtaining the real-time temperature information of the vehicle heating device according to the displayed image data, and then going to step S6;

[0072] Step S6, determine whether the detected temperature is greater than the threshold temperature, if yes, go to step S9; if no, go to step S7;

[0073] Step S7, transmit the alarm signal to the sound and light alarm, and go to step S8;

[0074] Step S8: The sound and light alarm is triggered, and a hypothermia alarm is issued, and the process goes to step S9;

[0075] Step S9, record the vehicle VIN, temperature value, determination result and other information into the database, display the real-time temperature information of the device to be heated on the monitoring system page, and issue an early warning to vehicles that do not meet the heating requirements, and go to step S10;

[0076] Step S10, determine whether the entire detection process is stopped, if yes, then end; if not, return to step S3, continue to obtain real-time infrared image and visible light image data from the infrared thermal imager, and enter the next round of cyclic detection process.

[0077] Each step is described in detail below.

[0078] In step S1, each function of the software is initialized, and the configured device parameters, alarm thresholds and other information are read. After the information is read correctly, the infrared image and visible light image acquisition devices are connected according to the IP addresses and device numbers of the infrared and visible light image acquisition devices to ensure normal communication between the software and the image acquisition devices. The infrared and visible light image acquisition devices in this embodiment can be infrared thermal imagers.

[0079] In step S2, according to the pre-configured DLL file path, the functions in the algorithm library are loaded into the memory to provide support for subsequent image processing and temperature calculation.

[0080] Step S3, see Figure 2 As shown, real-time infrared image and visible light image data are obtained from the infrared thermal imager.

[0081] Visible light images are RGB images. The RGB color model is a color standard in the industry. RGB represents the colors of the three channels of red, green, and blue. The RGB color model obtains a variety of colors by changing the three color channels of red, green, and blue and superimposing them on each other. This color standard includes almost all colors that can be perceived by human vision and is one of the most widely used color systems.

[0082] The embodiment of the present invention uses infrared and visible light cameras to obtain infrared images and visible light images of heating components of different vehicle models for the left and right rearview mirrors and rear windshield of the vehicle. Here, the infrared image is preferably a grayscale image, because the grayscale image can intuitively reflect the temperature distribution of the object being measured, and compared with other multi-channel color images, it occupies less memory and is more conducive to the subsequent model recognition efficiency. In practical applications, it is not limited to rearview mirrors and rear windshields.

[0083] In addition, traditional infrared images have low clarity, high correlation between channels of RGB images, poor color uniformity, and the human eye's recognition of images is not very ideal. Therefore, visual infrared images have poor visual effects and low clarity, are easily affected by environmental factors, and the detection standards are not quantified. At the same time, the detection results are difficult to trace, which is not convenient for subsequent quality tracing and fault handling analysis, and there is a high risk of defective products being outflowed.

[0084] Step S4, see Figure 3 As shown, the acquired infrared image and visible light image data are subjected to HSV color space image fusion processing, which specifically includes the following steps:

[0085] Step S401, calling the image processing function, performing channel separation processing of the acquired infrared image and visible light image data in the HSV color space respectively: converting the acquired infrared image into three channels of the HSV color space: H (Hue) channel, S (Saturation) channel, and V (Value) channel; converting the RGB color space of the acquired visible light image into three channels of the HSV color space: H channel, S channel, and V channel, and then going to step S402;

[0086] Step S402, mapping the H channel and the S channel separated from the infrared image and the visible light image in step S401 to the H channel and the S channel of the fused image respectively according to the grayscale of the infrared image;

[0087] The V channel separated from the infrared image in step S401 is fused with the V channel converted from the visible light image, and mapped to the V channel of the fused image;

[0088] Merge the V channel of the fused image with the H channel and S channel of the respectively mapped fused images to obtain the HSV color space of the fused image;

[0089] Convert the HSV color space of the fused image into the RGB color space to obtain an RGB image, and go to step S403;

[0090] Step S403: perform bilateral filtering on the RGB image obtained in step S402 to enhance the edge information of the target area and obtain a final fused image.

[0091] The following is a detailed explanation using a specific calculation formula as an example.

[0092] In step S401, the infrared image and the visible light image are converted into HSV color space respectively. HSV represents a subset of visible light in the color space including three elements: hue, saturation and value, which contains all colors in a certain color domain. The HSV system is based on the physiological understanding of color by the human eye, and can intuitively express the hue, saturation and brightness of the color. The independence between channels is strong, the image uniformity is good, and it is easy to process the image by channel, which is in line with the color perception of the human eye.

[0093] In step S402, according to the grayscale of the infrared image, the H channel and the S channel separated from the infrared image in step S401 are mapped to the H channel and the S channel of the fused image respectively. The specific calculation formulas are shown in (1) and (2).

[0094]

[0095] In formula (1), H(i, j) is any element in the pixel matrix on the H (Hue) channel of the fused image, i is the number of rows in the pixel matrix, j is the number of columns in the pixel matrix, and f IR (i, j) is the infrared V (Value, brightness) channel image f IR Any element in the pixel matrix, f IRmax is the maximum value of the infrared V channel image pixel matrix element, f IRmin It is the minimum value of the infrared V channel image pixel matrix element.

[0096]

[0097] In formula (2), S(i, j) is any element in the pixel matrix on the S channel of the fused image, k is the saturation coefficient, σ is the Gaussian standard deviation, and e is a natural constant in the formula, which is 2.718.

[0098] The V channel separated from the infrared image in step S401 is fused with the V channel converted from the visible light image and mapped to the V channel of the fused image. The specific calculation process is as follows:

[0099] Based on the difference between the infrared image and the visible light image V channel, the double threshold method is used to calculate the pixel values ​​of each part of the fused image. The specific calculation formulas are shown in (3) and (4):

[0100]

[0101] T2=f IRmax -fIRmin (4)

[0103] Here, the double thresholds are set as: the first threshold T1 and the second threshold T2, where T1 < T2. T1 represents the average standard value of the difference of a certain pixel in the V channel of the visible light and infrared images, and T2 represents the extreme difference value of the pixels in the V channel of the infrared image.

[0104] In formula (3), f VI (i, j) is any element in the pixel matrix of the visible light V channel image f VI in the pixel matrix.

[0105] Assume that V IR (i, j) is the value of an element in the pixel matrix of the V channel of the infrared image, and V VI (i, j) is the value of an element in the pixel matrix of the visible light V channel image, is the fusion coefficient of the infrared image, is the fusion coefficient of the visible light image. The following classifies different threshold intervals and discusses them separately as follows:

[0106] When |V IR (i, j) - V VI (i, j)| < T1, the difference between the pixel matrix values of the visible light and infrared image V channel images is small:

[0107]

[0108] When T1 ≤ |V IR (i, j) - V VI (i, j)| ≤ T2, there is a certain difference between the pixel matrix values of the visible light and infrared image V channel images:

[0109]

[0110] When |V IR (i, j) - V VI (i, j)| > T2, there is a large difference between the pixel matrix values of the visible light and infrared image V channel images:

[0111]

[0112] Assume that V(i, j) is the pixel value of the pixel matrix in the V channel after weighted fusion. The specific calculation formula is shown in (11):

[0113]

[0114] Combining the above classification discussion results, the pixel value of the pixel matrix in the V channel after weighted fusion is obtained through formula (11), and finally the fused image is converted into an RGB image, making the image have better target color and texture signals.

[0115] Step S403: bilateral filtering is performed on the RGB image obtained in step S402 to obtain a final fused image. The specific calculation process is as follows:

[0116] Bilateral filtering is a nonlinear filtering method based on spatial domain proximity and grayscale domain similarity. It can effectively preserve the high-frequency information of image edges, which is conducive to the effective segmentation of the edges of subsequent target areas. The specific calculation formula of bilateral filtering is as follows:

[0117]

[0118] In formula (12), f(x, y) represents the pixel value of the image point (x, y) after filtering, and D x,y is the adjacent pixel area of ​​the central coordinate point (k,l), (x,y) represents the coordinates of any pixel point in the image, (k,l) represents the coordinates of the center of the image, g(x,y) is the pixel value of each neighborhood, and w(x,y,k,l) ​​represents the weight coefficient of the grayscale domain weight and the spatial domain weight of the filter template at the central coordinate point (k,l).

[0119]

[0120] Formula (13) is the spatial domain weight kernel expression, w d (x, y, k, l) represents the value of the spatial proximity Gaussian function, and its weight decreases as the distance between the neighborhood pixel and the center point increases. d is the spatial domain standard deviation.

[0121]

[0122] Formula (14) is the grayscale domain weight kernel expression, w s (x, y, k, l) represents the Gaussian function value of pixel value similarity, and its weight decreases as the difference between the gray values ​​of neighboring pixels increases. s is the standard deviation of the grayscale domain; g(k,l) is the pixel value of the image at the center coordinate point (k,l).

[0123] w(x,y,k,l)=w d (x,y,k,l)·w s (x,y,k,l) ​​(15)

[0124] Formula (15) is the final calculated value of the Gaussian function, and w(x, y, k, l) represents the weight coefficient of the grayscale domain weight and the spatial domain weight of the filter template at the center coordinate point (k, l).

[0125] Based on the temperature measurement range and color scale of the infrared thermal imager, the hue, saturation and brightness of the corresponding image are calculated through color gamut transformation to obtain the infrared image and visible light image in the HSV color space. Then, the brightness channels of the infrared image and visible light image in the HSV color space are weighted fused, and bilateral filtering is used to enhance the edge information of the target area of ​​the image.

[0126] Image information fusion: Since the image feature information collected by a single sensor is not complete and cannot fully map the required multi-dimensional feature information, using multiple sensors can collect image feature information of different dimensions. By fusing the image feature information of different dimensions, the multi-dimensional feature information of the image can be obtained.

[0127] Step S5: Display the processed image data on the monitoring system page, and obtain the real-time temperature information of the vehicle heating device based on the displayed image data.

[0128] In this embodiment, an improved Yolov5s target detection model is used to obtain the real-time temperature information of the vehicle heating device. Yolov5s is a computer vision target detection model. The meaning of Yolov (You Only Look Once Version) is: all objects can be identified by scanning the image once. In order to ensure the real-time and accuracy of the detection of vehicle heating devices, to complete the recognition and detection of the temperature of vehicle heating devices of different models in real time and efficiently, and to facilitate mobile deployment and avoid the complexity of the network model, this application selects the Yolov5s network model with a smaller network size as the basic model for detection, and improves the structure of the network model to achieve the effectiveness and lightweight of the network model.

[0129] The main structure of the Yolov5s original model consists of input, backbone network, neck network, and head output. Among them, the neck network is responsible for multi-scale fusion of the features extracted by the backbone network and passes it to the prediction layer. At present, the infrared image itself has unclear edges and network features are difficult to capture. We choose to introduce CBAM (Convolutional Block Attention Module) to enhance the target feature expression ability; at the same time, the original model has losses in classification, confidence, and positioning. Considering the timeliness of actual production applications and the unclear edges of the image itself, α-CIoU replaces the original prediction box boundary loss function, combined with the CBAM module to improve the final image detection performance and model effectiveness.

[0130] In order to solve the problems of unclear edges and difficult to capture network features in infrared image recognition, a lightweight CBAM is introduced into the neck network. It is a module that combines space and channel attention mechanism. The weighted calculation of the output results of the convolution layer is performed through the channel attention module and the spatial attention module to obtain the final weighted result. The definition formulas of the channel attention mechanism and the spatial attention mechanism are as follows (16) and (17), thereby enhancing the ability to extract target features and improving detection performance.

[0131] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (16)

[0132] M s (F) = σ(f 7×7 ([AvgPool(F);MaxPool(F)])) (17)

[0133] Among them, M C (F) is the channel attention value, M S (F) is the spatial attention value, AvgPool(F) is the average pooling feature vector, MaxPool(F) is the maximum pooling feature vector, σ is the S-shaped growth curve sigmoid activation function, and its monotonically increasing characteristic can be used as the threshold function of the neural network to map the variable to (0, 1). MLP represents the weight sharing mechanism, which essentially uses the same weights for the channel feature map, and can use the least convolution parameters; f 7*7 It means that a convolution operation using a 7*7 convolution kernel can extract multiple features of an image.

[0134] In order to improve the regression speed and accuracy of the prediction box and ensure the effectiveness of the detection model, α-CIoU is used to replace the original prediction box boundary loss function to effectively promote the network gradient update rate. α-CIoU is a loss function used in target detection tasks. It improves the IoU loss value of the original prediction box boundary and comprehensively considers the position, size and overlap rate of the target box and the real box. Its definition formulas are as follows (18), (19), and (20).

[0135]

[0136]

[0137]

[0138] Among them, l α-CIoU is the loss function value, gt represents the manually annotated target real box, ρ 2α (b,b gt ) is the center point b of the predicted box and the center point b of the real boxgt The Euclidean distance of the predicted box and the real box; c is the diagonal length of the minimum closed area that can contain the predicted box and the real box, α is an adjustable parameter used to balance the impact of the center point distance and the aspect ratio difference on the similarity, β is the weight parameter, v is the similarity of the aspect ratio, w is the real box width, h is the real box height, w gt is the prediction box width, h gt is the predicted box height.

[0139] An improved target detection model is built based on the pytorch framework. A lightweight CBAM attention module is introduced into the neck network. The α-CIoU replaces the original prediction box boundary loss function to improve the target feature extraction capability, the prediction box regression speed and accuracy. The infrared training set is used for training to identify the positions of the left and right rearview mirrors and rear windshield of the vehicle.

[0140] Based on the predicted infrared image detection area, the image and temperature width bar are gray-scaled, and the temperature of each pixel is extracted by interpolation method, and the highest and lowest temperatures in the selected frame area are obtained. Then, the Kalman filter method is used to optimize the measured temperature value to obtain a stable and reliable temperature output T, see Figure 4 Mid-infrared images and grayscale images identify the temperature values ​​of each area. Figure 4 The three temperature values ​​displayed are: the measured temperatures of three points of the left and right rearview mirrors and the rear windshield, and the specific temperatures are: 38.1℃, 38.2℃, and 38.5℃ respectively.

[0141] The measured temperature is compared with the threshold temperature. If the measured temperature T is greater than the threshold temperature T3, the heating function of the electrical appliance is determined to be qualified, and the qualified information is displayed on the RGB image.

[0142] T3=T4+t(21)

[0143] Wherein, T3 represents the threshold temperature, T4 represents the current room temperature, and t represents the theoretical temperature rise generated by heating.

[0144] See also Figure 5 As shown, in step S5, based on the input of the fused image data set, the yolov5s target detection algorithm under the pytorch framework is improved to recognize temperature, so as to improve the accuracy of the output final temperature recognition result.

[0145] The infrared temperature recognition algorithm with improved yolov5s network structure mainly includes introducing a lightweight CBAM attention module in the neck network, obtaining weighted calculation results through the channel attention module and the spatial attention module, enhancing the ability to extract target features, and improving the detection performance. At the same time, the original prediction box boundary loss function is replaced by α-CIoU, which effectively improves the convergence speed and accuracy of the prediction box, thereby ultimately realizing the effective and accurate perception prediction of the heating temperature of the rearview mirror and rear windshield during vehicle function detection.

[0146] In step S6, it is determined whether the detected temperature is greater than a preset threshold temperature. If yes, go to step S9; if no, go to step S7;

[0147] Step S7, display NG, indicating failure, the alarm signal is transmitted to the sound and light alarm, and go to step S8;

[0148] Step S8: The sound and light alarm is triggered, and a hypothermia alarm is issued, and the process goes to step S9;

[0149] Step S9, display OK, indicating qualified, record the vehicle VIN, temperature value, judgment result and other information into the database, display the real-time temperature information of the device to be heated on the monitoring system page, issue early warning prompts for vehicles that do not meet the heating requirements, and realize early warning prompts and quality tracking and tracing of vehicles that have not heated up.

[0150] On the other hand, see Figure 6 As shown, an embodiment of the present invention further provides a device heating detection system based on HSV color space image fusion, including:

[0151] The HSV conversion unit is used to convert the infrared image and visible light image data of the device to be heated into the HSV color space for channel separation;

[0152] The weighted fusion unit is used to: perform weighted fusion on the brightness channels of infrared and visible light images;

[0153] RGB conversion unit, used to: convert the fused HSV color space into an RGB image;

[0154] A bilateral filtering unit, used to: perform bilateral filtering on the RGB image to obtain a fused image;

[0155] The temperature detection unit is used to: calculate the fused image based on the temperature recognition algorithm of the target detection model to obtain the real-time temperature information of the device to be heated.

[0156] Further, the HSV conversion unit is specifically used for:

[0157] The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

[0158] Furthermore, the weighted fusion unit is specifically used for:

[0159] Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively;

[0160] The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

[0161] Furthermore, the temperature detection unit uses a temperature recognition algorithm based on a target detection model as follows:

[0162] A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once;

[0163] The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

[0164] See also Figure 7 As shown, the device heating detection system based on HSV color space image fusion further includes:

[0165] The display warning unit is used to: after obtaining the real-time temperature information of the device to be heated, display the real-time temperature information of the device to be heated, record the vehicle identification code VIN code, and issue a warning prompt to the vehicle that does not meet the heating requirements.

[0166] Among them, the functional implementation of each module in the above-mentioned device heating detection system based on HSV color space image fusion corresponds to the steps in the above-mentioned device heating detection method embodiment based on HSV color space image fusion, and its functions and implementation process will not be repeated here one by one.

[0167] Traditional infrared cameras distinguish targets and backgrounds by emitting electronic beams of different colors from targets, but the image itself has very low clarity. At the same time, RGB images have problems such as poor color uniformity and unclear target edges. Background information is also basically lost, which is not conducive to the subsequent target area segmentation and temperature extraction of the heating device. Therefore, this application performs weighted fusion processing on infrared images and visible light images based on the HSV color space to enhance the image target color and texture signals, and combines bilateral filtering with the yolov5s algorithm to effectively preserve image edge information and ensure the subsequent image segmentation effect.

[0168] In actual applications, the host computer reads visible light and infrared image data through the TCP / IP protocol, calls the image fusion processing algorithm, displays the vehicle's infrared image in real time, identifies the temperature range, and based on data processing technology, records the vehicle's VIN code, temperature value, and judgment result information into the database, thereby realizing early warning prompts and quality tracking and tracing of vehicles that have not heated up.

[0169] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0170] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0171] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A device heating detection method based on HSV color space image fusion, characterized in that: The following steps are involved: The infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation; Perform weighted fusion of the brightness channels of infrared and visible light images; Converting the fused HSV color space into an RGB image, and performing bilateral filtering on the RGB image to obtain a fused image; A temperature recognition algorithm based on the target detection model is used to calculate the fused image and obtain the real-time temperature information of the device to be heated.

2. The device heating detection method based on HSV color space image fusion according to claim 1, characterized in that: The infrared image and visible light image data of the device to be heated are converted into the HSV color space for channel separation, which specifically includes the following steps: The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

3. The device heating detection method based on HSV color space image fusion according to claim 2, characterized in that: The weighted fusion of the brightness channels of the infrared and visible light images specifically includes the following steps: Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively; The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

4. The device heating detection method based on HSV color space image fusion according to claim 3, characterized in that: The temperature recognition algorithm based on the target detection model is: A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once; The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

5. The device heating detection method based on HSV color space image fusion according to claim 4, characterized in that: After obtaining the real-time temperature information of the device to be heated, the following steps are also included: On the monitoring system page, the real-time temperature information of the heated device is displayed, the vehicle identification code VIN code is recorded, and early warning prompts are issued for vehicles that do not meet the heating requirements.

6. A device heating detection system based on HSV color space image fusion, characterized in that: include: The HSV conversion unit is used to convert the infrared image and visible light image data of the device to be heated into the HSV color space for channel separation; The weighted fusion unit is used to: perform weighted fusion on the brightness channels of infrared and visible light images; RGB conversion unit, used to: convert the fused HSV color space into an RGB image; A bilateral filtering unit, used to: perform bilateral filtering on the RGB image to obtain a fused image; The temperature detection unit is used to: calculate the fused image based on the temperature recognition algorithm of the target detection model to obtain the real-time temperature information of the device to be heated.

7. The device heating detection system based on HSV color space image fusion according to claim 6, characterized in that: The HSV conversion unit is specifically used for: The infrared image and visible light image data of the device to be heated are all converted into the hue H channel, saturation S channel, and brightness V channel of the HSV color space.

8. The device heating detection system based on HSV color space image fusion according to claim 7, characterized in that: The weighted fusion unit is specifically used for: Map the H channel and S channel of the infrared image and visible light image to the H channel and S channel of the fused image respectively; The V channel converted from the infrared image is fused with the V channel converted from the visible light image to obtain the V channel of the fused image.

9. The device heating detection system based on HSV color space image fusion according to claim 8, characterized in that: The temperature detection unit adopts the temperature recognition algorithm based on the target detection model as follows: A lightweight attention module CBAM is introduced into the neck network of the yolov5s network structure. The meaning of yolov is: all objects can be identified by scanning the image once; The weighted calculation results are obtained through the channel attention module and the spatial attention module, and the α-CIoU loss function is used to replace the original prediction box boundary loss function.

10. The device heating detection system based on HSV color space image fusion according to claim 9, characterized in that: The system further comprises: The display warning unit is used to: after obtaining the real-time temperature information of the device to be heated, display the real-time temperature information of the device to be heated, record the vehicle identification code VIN code, and issue a warning prompt to the vehicle that does not meet the heating requirements.

Images