Method and system for measuring room temperature of thermal users based on temperature measurement images captured by smart devices

Abstract

The present invention discloses a method for measuring the room temperature of a heating user based on capturing temperature measurement images using an intelligent device, comprising: the heating user obtains room temperature measurement values ​​at different room temperature measurement locations in real time through a room temperature measurement device within different preset time periods; an intelligent terminal device is used to collect images of the multiple room temperature measurement values, which are then sent to a room temperature monitoring host computer via a communication module, and an incentive result is generated according to preset rules to encourage the user to measure the room temperature; the room temperature monitoring host computer preprocesses, segments, and extracts labels from the room temperature images obtained in real time, and then inputs the images into a constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement location of the heating user on the image and identifying the corresponding room temperature measurement value. The present invention does not require going to the user's home to measure the room temperature, thus reducing the communication costs incurred by existing wireless room temperature collection. It can also ensure the validity of the measurement location, provide reliable room temperature data for heating network scheduling, and save operating costs.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart heating room temperature measurement, and specifically relates to a method and system for measuring the room temperature of a heating user based on temperature measurement images taken by a smart device. Background Art

[0002] In the heating industry, the arrival of the centralized heating season creates numerous heating conflicts. Heating companies seek to save costs, while residents frequently complain about substandard indoor temperatures and request temperature measurements. Currently, many property management companies and heating companies rely on manual door-to-door temperature measurements during the heating season, requiring extensive travel, a heavy workload, and untimely data collection.

[0003] In the centralized heating sector, wireless room temperature measurement has become a popular temperature measurement method in recent years. With the continuous advancement of computer, electronics, and communications technologies, automation is accelerating across various industries, leading to higher requirements for real-time data collection, reliability, and information volume. Wireless room temperature measurement offers the advantage of enabling centralized monitoring of multiple ambient temperatures across existing buildings where wiring is impractical, enabling control of air conditioning and heating systems, thereby achieving energy savings and efficiency gains. Heating companies use this temperature data to adjust energy consumption, ensuring heating quality while conserving energy.

[0004] However, wireless room temperature monitors use wireless public networks for data transmission. A large number of wireless room temperature monitors will inevitably generate a large amount of communication costs. This communication cost confuses heating companies and also limits the large-scale adoption of wireless room temperature monitors. In addition, there are telephone line temperature measurement and GPRS temperature measurement. These two methods have their own disadvantages. The telephone line thermometer needs to occupy the user's home phone, which brings inconvenience to the user's daily life. The GPRS temperature measurement requires the use of China Mobile's GPRS network and traffic metering, which brings inconvenience to the user's subsequent maintenance. Moreover, to measure the heating temperature, the room temperature meter should be placed in the center of the measured room one meter from the ground with the doors and windows closed normally for more than one hour. The stable reading of the meter is the actual heating temperature. However, some malicious users will collect indoor temperatures at the door, window, refrigerator or other abnormal locations, making the room temperature data illegal and invalid and damaging the interests of the heating company.

[0005] Based on the above technical problems, it is necessary to design a new method for measuring the room temperature of thermal users based on temperature measurement images taken by smart devices. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for measuring the room temperature of heat users based on temperature measurement images taken by smart devices, thereby improving the reliability of room temperature measurement, providing detailed and reliable heating effect data for heat network scheduling, reducing the communication costs generated by existing wireless room temperature collection, and realizing economical operation of the heat network.

[0007] In order to solve the above technical problems, the present invention provides a method for measuring the room temperature of a thermal user based on a temperature measurement image captured by a smart device, which comprises:

[0008] Step S1: The heat user obtains room temperature measurement values ​​at different room temperature measurement positions in real time through a room temperature measurement device within different preset time periods;

[0009] Step S2: Using an intelligent terminal device to collect images of multiple room temperature measurements, the images are sent to a room temperature monitoring host computer via a communication module, and an incentive result is generated according to a preset rule to encourage the user to measure the room temperature;

[0010] In step S3, the room temperature monitoring host computer preprocesses, segments, and extracts labels on the room temperature image acquired in real time, and then inputs the image into the constructed convolutional neural network model for image recognition to determine the validity of the room temperature measurement position of the thermal user in the image and identify the corresponding room temperature measurement value.

[0011] Furthermore, the step S1 specifically includes:

[0012] For the heat users in the same cell, different temperature measurement times in the morning, noon and evening are selected as the first time period T1, the second time period T2 and the third time period T3, which are expressed as T = [T1, T2, T3];

[0013] The thermal user selects at least one valid room temperature measurement location, represented as M = [M1, M2, ..., M i ], i≥1;

[0014] The room temperature measurement device is used to measure the room temperature at different room temperature measurement positions in each time period to obtain the corresponding room temperature measurement values, which are expressed as and

[0015] Furthermore, in step S2, the intelligent terminal device is used to respectively capture images of multiple room temperature measurement values ​​and transmits the images to the room temperature monitoring host computer through the communication module, and generates an incentive result according to a preset rule to encourage the user to measure the room temperature, which specifically includes:

[0016] The intelligent terminal device uses its camera unit to measure the room temperature values ​​of different room temperature measurement positions within the first time period t1. Room temperature measurement values ​​at different room temperature measurement positions in the second time period t2 and the room temperature measurement values ​​at different room temperature measurement positions in the third time period t3 Shoot and obtain the original room temperature image data set, and send it to the room temperature monitoring host computer in sequence within the corresponding time period;

[0017] Among them, when the thermal user uploads the corresponding room temperature image data in sequence within the preset first time period T1, the second time period T2 and the third time period T3, an incentive result is generated according to the preset rules to encourage the user. The preset rules are: generate electronic red envelopes with reference to the red envelope generation mechanism of the WeChat platform; or preset a prize list and randomly generate prizes by lottery.

[0018] Furthermore, in step S3, the room temperature monitoring host computer preprocesses the acquired original room temperature image data set, including:

[0019] Convert image signals into digital signals through acquisition and quantification;

[0020] Image enhancement of room temperature images is performed through grayscale correction and grayscale transformation;

[0021] The room temperature image is denoised and smoothed by local smoothing and bilateral filtering methods;

[0022] The room temperature image dataset is subjected to image cropping, standardization and normalization in sequence;

[0023] The calculation formula for the image standardization process is: μ is the mean of the image, x is the image matrix, σ is the standard deviation, and p is the number of pixels in the image.

[0024] Furthermore, the image segmentation in step S3 specifically includes:

[0025] After calculating the mean, variance, and grayscale value of the preprocessed input room temperature image, the optimal threshold for binary segmentation is calculated based on the mean and variance of the image. All pixels in the image with grayscale values ​​greater than the optimal threshold are marked as the target area, and all pixels in the image with grayscale values ​​less than the optimal threshold are marked as the background area. The calculation formula used is:

[0026]

[0027] Where, the background area is represented by 0, the target area is represented by 1, F(I,J) represents the input room temperature image, G(I,J) represents the output room temperature image, pixels are represented by i,j, the grayscale value of the input room temperature image is represented by f(i,j), the grayscale value of the output image is represented by g(i,j), and the optimal threshold is represented by T.

[0028] Furthermore, the image segmentation also includes segmenting the target area of ​​the image into image characters and spatial positions, specifically including:

[0029] A deep learning image algorithm is used to build an image character recognition model and a spatial position model for identifying input room temperature images;

[0030] The room temperature image is detected and located by a detection algorithm to obtain the room temperature character area and non-character area, and the size of the sliding window is set respectively;

[0031] Sliding a sliding window from left to right within the room temperature character area to form a series of windows, and inputting the image area within the window into the image character recognition model to determine the probability of whether a single character is included in the window, and then optimizing the series of windows and their corresponding character probabilities, and the retained windows are the characters in the room temperature image;

[0032] The sliding window is slid from left to right in the non-character area of ​​the image to form a series of windows, and the image area in the window is input into the spatial position model. It is determined whether the window includes multiple position nodes based on the pre-established two-dimensional image position node information library, and the retained window is the measurement space position in the room temperature image.

[0033] Furthermore, the tag extraction in step S3 specifically includes:

[0034] Taking the multiple segmented character images and spatial position images in the room temperature image as a set of images to be extracted, and extracting a convolution feature map of each image to be extracted by using a convolution layer in a convolutional neural network;

[0035] In a feature mapping layer of a convolutional neural network, a plurality of extraction regions are set for each convolution feature map, and a region category and region information of each extraction region are used as respective region labels; the region categories include a room temperature measurement device region, a room temperature data region, and a room temperature measurement position region, and the region information includes at least digital character information and room temperature measurement position spatial information;

[0036] Taking the region labels of all extracted regions corresponding to each convolution feature map as the label set of the convolution feature map, collecting the label sets of all convolution feature maps, and obtaining a room temperature label set, wherein the room temperature label set at least includes digital character information and room temperature measurement position spatial information;

[0037] Among them, the indoor measurement position images of thermal users are collected in advance and the corresponding label identification points are extracted. The label identification points exclude abnormal identification points such as windows, doorways and positions with cooling effects, and retain position identification points with large weight values ​​and obvious features.

[0038] Furthermore, in step S3, the construction of the convolutional neural network model specifically includes:

[0039] During a historical time period, the image dataset processed by steps S1 and S2 is collected as sample data, the room temperature image set after preprocessing and image segmentation of the sample images is used as input variables, including a training set and a test set, and the room temperature label set is used as the output variable;

[0040] Constructing a convolutional neural network, the convolutional neural network comprising an input layer, multiple convolutional layers and multiple pooling layers, multiple fully connected layers, and an output layer, setting a number of training times and a training target error, and training the convolutional neural network using room temperature images in the training set so that a feature value of the output layer of the convolutional neural network is the same as the label information of each input room temperature image; wherein the size of the input layer is the same as the predetermined size of the room temperature images in the training set;

[0041] After the convolutional neural network training is completed, the convolutional neural network is tested using room temperature images in the test set. When the prediction error of the convolutional neural network in the test set is less than a preset value, the test is passed and the convolutional neural network can be used for room temperature measurement.

[0042] Furthermore, in step S3, the input is fed into the constructed convolutional neural network model for image recognition, determining the validity of the room temperature measurement position of the thermal user on the image and identifying the corresponding room temperature measurement value, specifically including:

[0043] After the real-time user room temperature measurement image is subjected to the preprocessing, image segmentation and label extraction, it is input into the constructed convolutional neural network model to determine whether the room temperature measurement position of the hot user on the image is consistent with the preset room temperature target position, and then judge the validity of the spatial position. When the spatial position is valid, the corresponding room temperature measurement value is identified and read.

[0044] The second aspect of the present invention further provides a user room temperature measurement system based on a temperature measurement image captured by a smart device, comprising:

[0045] The room temperature acquisition module is used for users to obtain room temperature measurement values ​​at different room temperature measurement positions in real time through the room temperature measurement device within different preset time periods;

[0046] An image acquisition module is used to use an intelligent terminal device to respectively acquire images of multiple room temperature measurement values ​​and then send them to a room temperature monitoring host computer through a communication module;

[0047] A user incentive module, used to generate incentive results according to preset rules to encourage users to measure room temperature;

[0048] The image recognition module is used to preprocess, segment, and extract labels from the room temperature images acquired in real time by the room temperature monitoring host computer, and then input them into the constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement position of the thermal user in the image and identifying the corresponding room temperature measurement value.

[0049] The beneficial effects of the present invention are:

[0050] (1) The present invention uses a room temperature measuring device to obtain room temperature measurement values ​​at different room temperature measurement positions in real time during different preset time periods by a heat user; an intelligent terminal device is used to collect images of multiple room temperature measurement values ​​and then transmits them to a room temperature monitoring host computer through a communication module; the room temperature monitoring host computer pre-processes, segments and extracts labels of the room temperature images obtained in real time, and then inputs them into a constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement position of the heat user on the image and identifying the corresponding room temperature measurement value; the room temperature measuring device can be an ordinary mercury meter, and the room temperature measurement data is photographed, uploaded and pre-processed, segmented and extracted by an intelligent terminal device (mobile phone), and then a convolutional neural network is used to determine the validity of the room temperature measurement position and read the value, thereby reducing the communication costs generated by existing wireless room temperature collection, and eliminating the need to frequently go to the user's home to measure the room temperature, while ensuring the reliability and effectiveness of the room temperature measurement position. The room temperature monitoring host computer obtains a large amount of timely indoor temperature information, thereby providing detailed and reliable heating effect data for the heat network dispatch, realizing the economic operation of the heat network, and saving a lot of operating costs for the heat enterprise;

[0051] (2) The present invention generates incentive results according to preset rules to encourage users to measure room temperature. The preset rules are to generate electronic red envelopes with reference to the red envelope generation mechanism of the WeChat platform, or to preset a prize list and randomly generate prizes in a lottery manner, thereby effectively motivating heat users to actively participate in room temperature measurement, making it convenient for heat companies to understand the room temperature data of heat users so as to adjust the heating supply;

[0052] (3) The present invention segments the room temperature image into image characters and measurement space positions, extracts corresponding labels, constructs a convolutional neural network model to recognize the room temperature image, determines the validity of the measurement position and reads the corresponding room temperature value, thereby effectively improving the reliability of room temperature measurement.

[0053] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 This is a flow chart of a method for measuring room temperature of a thermal user based on temperature measurement images captured by a smart device according to the present invention;

[0057] Figure 2 This is a functional block diagram of a thermal user room temperature measurement system based on temperature measurement images captured by a smart device according to the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] Example 1

[0060] Figure 1 This is a flow chart of the method for measuring the room temperature of a thermal user based on temperature measurement images captured by a smart device according to the present invention.

[0061] like Figure 1 As shown, this embodiment 1 provides a method for measuring the room temperature of a thermal user based on capturing a temperature measurement image using a smart device, which includes:

[0062] Step S1: The heat user obtains room temperature measurement values ​​at different room temperature measurement positions in real time through a room temperature measurement device within different preset time periods;

[0063] Step S2: Using an intelligent terminal device to collect images of multiple room temperature measurements, the images are sent to a room temperature monitoring host computer via a communication module, and an incentive result is generated according to a preset rule to encourage the user to measure the room temperature;

[0064] In step S3, the room temperature monitoring host computer preprocesses, segments, and extracts labels on the room temperature image acquired in real time, and then inputs it into the constructed convolutional neural network model for image recognition to determine the validity of the room temperature measurement position of the thermal user in the image and identify the corresponding room temperature measurement value.

[0065] In this embodiment, step S1 specifically includes:

[0066] For the heat users in the same cell, different temperature measurement times in the morning, noon and evening are selected as the first time period T1, the second time period T2 and the third time period T3, which are expressed as T = [T1, T2, T3];

[0067] The thermal user selects at least one valid room temperature measurement location, represented as M = [M1, M2, ..., M i ], i≥1;

[0068] The room temperature measurement device is used to measure the room temperature at different room temperature measurement positions in each time period to obtain the corresponding room temperature measurement values, which are expressed as and

[0069] It should be noted that the room temperature measurement device is generally not required to have a communication function. On the one hand, it eliminates the need for a communication module and related costs, and on the other hand, it protects user privacy and is highly secure and not easily damaged.

[0070] In this embodiment, step S2 specifically includes:

[0071] The intelligent terminal device uses its camera unit to measure the room temperature values ​​of different room temperature measurement positions within the first time period t1. Room temperature measurement values ​​at different room temperature measurement positions in the second time period t2 and the room temperature measurement values ​​at different room temperature measurement positions in the third time period t3 Shoot and obtain the original room temperature image data set, and send it to the room temperature monitoring host computer in sequence within the corresponding time period;

[0072] Among them, when the thermal user uploads the corresponding room temperature image data in sequence within the preset first time period T1, the second time period T2 and the third time period T3, an incentive result is generated according to the preset rules to encourage the user. The preset rules are: generate electronic red envelopes with reference to the red envelope generation mechanism of the WeChat platform; or preset a prize list and randomly generate prizes by lottery.

[0073] In this embodiment, in step S3, the room temperature monitoring host computer preprocesses the acquired raw room temperature image dataset, including:

[0074] Convert image signals into digital signals through acquisition and quantification;

[0075] Image enhancement of room temperature images is performed through grayscale correction and grayscale transformation;

[0076] The room temperature image is denoised and smoothed by local smoothing and bilateral filtering methods;

[0077] The room temperature image dataset is subjected to image cropping, standardization and normalization in sequence;

[0078] The calculation formula for image standardization is: μ is the mean of the image, x is the image matrix, σ is the standard deviation, and p is the number of pixels in the image.

[0079] In practical applications, preprocessing also includes image rotation. Changing the spatial relationship between pixels in the image is called image geometric transformation, which in turn changes the spatial position of the image. The pixel points before and after the transformation can be obtained. The calculation formula is:

[0080]

[0081] The output image pixel position coordinates are represented by x and y in the above formula, and the input image pixel position coordinates are represented by x0 and y0. The corresponding relationship between them is represented by U and V, that is, the corresponding x and y coordinates are calculated by the input x0 and y0.

[0082] In addition, the image can be rotated at any angle in theory, such as 90 degrees, 180 degrees and 270 degrees. Fixed integer angles are used in counterclockwise rotation.

[0083] In this embodiment, in step S3, image segmentation is performed on the room temperature image dataset, specifically including:

[0084] After calculating the mean, variance, and grayscale value of the preprocessed input room temperature image, the optimal threshold for binary segmentation is calculated based on the mean and variance of the image. All pixels in the image with grayscale values ​​greater than the optimal threshold are marked as the target area, and all pixels in the image with grayscale values ​​less than the optimal threshold are marked as the background area. The calculation formula used is:

[0085]

[0086] Where, the background area is represented by 0, the target area is represented by 1, F(I,J) represents the input room temperature image, G(I,J) represents the output room temperature image, pixels are represented by i,j, the grayscale value of the input room temperature image is represented by f(i,j), the grayscale value of the output image is represented by g(i,j), and the optimal threshold is represented by T.

[0087] In this embodiment, image segmentation further includes segmenting the target area of ​​the image into image characters and spatial positions, specifically including:

[0088] A deep learning image algorithm is used to build an image character recognition model and a spatial position model for identifying input room temperature images;

[0089] The room temperature image is detected and located by a detection algorithm to obtain the room temperature character area and non-character area, and the size of the sliding window is set respectively;

[0090] The sliding window is traversed from left to right in the room temperature character area to form a series of windows. The image area within the window is input into the image character recognition model to determine the probability of whether the window contains a single character. The series of windows and their corresponding character probabilities are then optimized. The retained windows are the characters in the room temperature image.

[0091] The sliding window is traversed from left to right in the non-character area of ​​the image to form a series of windows, and the image area within the window is input into the spatial position model. According to the pre-established two-dimensional image position node information library, it is determined whether the window includes multiple position nodes. The retained window is the measurement space position in the room temperature image.

[0092] In this embodiment, in step S3, label extraction is performed on the room temperature image dataset, specifically including:

[0093] The multiple segmented character images and spatial position images in the room temperature image are used as a set of images to be extracted, and the convolutional layer in the convolutional neural network is used to extract the convolutional feature map of each image to be extracted;

[0094] In a feature mapping layer of a convolutional neural network, a plurality of extraction regions are set for each convolution feature map, and a region category and region information of each extraction region are used as respective region labels; the region categories include a room temperature measurement device region, a room temperature data region, and a room temperature measurement location region, and the region information includes at least digital character information and room temperature measurement location spatial information;

[0095] Taking the region labels of all extracted regions corresponding to each convolution feature map as the label set of the convolution feature map, collecting the label sets of all convolution feature maps, and obtaining a room temperature label set, wherein the room temperature label set at least includes digital character information and room temperature measurement position spatial information;

[0096] Among them, the indoor measurement position images of heat users are collected in advance and the corresponding label identification points are extracted. The label identification points exclude abnormal identification points such as windows, doorways and positions with cooling effects, and retain position identification points with large weight values ​​and obvious features. The size of the weight value is preset by the system, and the position identification points are based on the corresponding identification features preset by the system to facilitate feature recognition and label extraction.

[0097] In this embodiment, in step S3, the construction of the convolutional neural network model specifically includes:

[0098] During the historical time period, the image dataset processed by steps S1 and S2 is collected as sample data, the room temperature image set after preprocessing and image segmentation of the sample images is used as input variables, including a training set and a test set, and the room temperature label set is used as the output variable;

[0099] Constructing a convolutional neural network, the convolutional neural network includes an input layer, multiple convolutional layers and multiple pooling layers, multiple fully connected layers, and an output layer, setting the number of training times and the training target error, and training the convolutional neural network using room temperature images in the training set so that the eigenvalues ​​of the convolutional neural network output layer are the same as the label information of each input room temperature image; wherein the size of the input layer is the same as the predetermined size of the room temperature images in the training set;

[0100] After the convolutional neural network training is completed, the convolutional neural network is tested using room temperature images in the test set. When the prediction error of the convolutional neural network in the test set is less than a preset value, the test is passed and the convolutional neural network can be used for room temperature measurement.

[0101] In actual applications, the convolutional neural network includes an input layer, a first convolutional layer, a second pooling layer, a third convolutional module group, a fourth pooling layer, a fifth convolutional layer, a sixth pooling layer, a seventh convolutional layer, an eighth fully connected layer, and an output layer, which are connected in sequence for model training; wherein, the third convolutional module group includes three branches, including a first branch, a second branch, and a third branch; the first branch includes a first branch first convolutional layer and a first branch second convolutional layer; the second branch includes a second branch first convolutional layer and a second branch second convolutional layer; the third branch includes a third branch convolutional layer; through the preset convolution parameters, after each layer performing convolution processing obtains the image of the upper layer for convolution processing, the number of feature images obtained shows a gradient increasing trend; through the preset pooling parameters, after each layer performing pooling processing obtains the image of the upper layer for pooling processing, the size of the feature image obtained shows a gradient decreasing trend; the eighth fully connected layer is preset with a number of neurons, each neuron is connected to the output layer for outputting label results.

[0102] Each convolutional neural network layer uses a preset activation function for convolution processing. The activation function uses the ReLU activation function, expressed as f(x) = max(0, x), where x represents the output of each convolutional layer. The output layer of the convolutional neural network uses the Softmax function to obtain the probability of the image's category and selects the maximum probability as the final output.

[0103] In this embodiment, in step S3, the image is input into the constructed convolutional neural network model for image recognition, and the validity of the room temperature measurement position of the thermal user on the image is determined and the corresponding room temperature measurement value is identified, which specifically includes:

[0104] The real-time user room temperature measurement images are preprocessed, segmented, and labeled, and then input into the constructed convolutional neural network model to determine whether the room temperature measurement position of the thermal user on the image is consistent with the preset room temperature target position, and then judge the validity of the spatial position. If the spatial position is valid, the corresponding room temperature measurement value is identified and read.

[0105] It should be noted that the determination of the spatial position mainly includes determining whether the room temperature measurement device is placed in an abnormal position such as a windowsill, door, refrigerator, etc. If it is in an abnormal position, the room temperature measurement image is invalid.

[0106] In actual applications, the reading of room temperature measurement values ​​is realized through the digital recognition algorithm framework. Usually, the steps of the recognition algorithm are divided into digital extraction, template matching and other steps. Optimization processing is performed during digital template matching. The combination of contour and projection is used to enhance the compatibility of digital recognition. The area of ​​interest of digital characters is horizontally and vertically projected. The digital features first store the features of the series of numbers on the thermometer as a template, and then match the digital extraction and template on the input image one by one. The one with the highest similarity is the same number. By verifying this method, good matching results can be obtained. The distance between digital characters is judged. If the distance is less than 50% of the width of the number, it is judged to be the same number and combined into a two-digit number; if the distance is greater than 50% of the width of the number, it is judged to be two numbers and calculated separately; if the room temperature measuring device is a thermometer, scale recognition is required, including scale extraction, scale lines in the thermometer image They are all in the vertical direction, but because it is a glass thermometer, the reflection, refraction and penetration of the glass into the background are not uniform, and the integrity of the scale is not strong. Therefore, the scale style and position are corrected twice in combination with the characteristics of vertical projection and uniform distribution of multiple scale positions to obtain a more ideal image; mercury column recognition: Sobel is used to extract pixels in the horizontal direction and combined with Hough transform to obtain more accurate mercury column pixel coordinates; mercury thermometer reading calculation: the corresponding scale and number are extracted, and the scale, number, and liquid column head are formed into an associative array. All long scales and a group of long scales closest to the liquid column head are extracted to form an associative group A, and then the scales at the corresponding positions of all digital areas are extracted to form an associative group B. The scales at the corresponding positions of all numbers at both ends of the liquid column are extracted to form an associative group C. The distance pixel length between the two is obtained. In addition, the temperature difference between the two values ​​is 2 degrees, and the pixel equivalent value can be obtained. The actual liquid column reading is obtained based on the distance and pixel equivalent value.

[0107] Example 2

[0108] Figure 2 This is a principle block diagram of a thermal user room temperature measurement system based on temperature measurement images captured by a smart device, according to the present invention.

[0109] like Figure 2 As shown, this embodiment 2 proposes a user room temperature measurement system based on a temperature measurement image captured by a smart device, which includes:

[0110] The room temperature acquisition module is used for users to obtain room temperature measurement values ​​at different room temperature measurement positions in real time through the room temperature measurement device within different preset time periods;

[0111] An image acquisition module is used to use an intelligent terminal device to respectively acquire images of multiple room temperature measurement values ​​and then send them to a room temperature monitoring host computer through a communication module;

[0112] A user incentive module, used to generate incentive results according to preset rules to encourage users to measure room temperature;

[0113] The image recognition module is used for room temperature monitoring. The host computer preprocesses, segments and extracts labels of the room temperature images acquired in real time, and then inputs them into the constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement position of the thermal user in the image and identifying the corresponding room temperature measurement value.

[0114] The present invention enables a heat user to obtain room temperature measurement values ​​at different room temperature measurement positions in real time through a room temperature measuring device within different preset time periods; an intelligent terminal device is used to respectively perform image acquisition on multiple room temperature measurement values ​​and then transmit the images to a room temperature monitoring host computer through a communication module; the room temperature monitoring host computer pre-processes, segments and extracts labels on the room temperature images acquired in real time, and then inputs the images into a constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement position of the heat user on the image and identifying the corresponding room temperature measurement value; an ordinary mercury meter can be used as the room temperature measuring device, and an intelligent terminal device (mobile phone) is used to take a photo of the room temperature measurement data, upload and pre-process the images, segment the images and extract labels, and then a convolutional neural network is used to determine the validity of the room temperature measurement position and read the values, thereby reducing the communication costs generated by existing wireless room temperature acquisition, and eliminating the need to frequently go to the user's home to measure the room temperature, and further ensuring that the room temperature measurement position is accurate. The device is reliable and effective, and the room temperature monitoring host computer obtains a large amount of timely indoor temperature information, thereby providing detailed and reliable heating effect data for the heating network dispatch, realizing the economic operation of the heating network, and saving a lot of operating costs for the heating company; the present invention generates incentive results according to preset rules to encourage users to measure the room temperature, and generates incentive results according to preset rules to encourage users. The preset rules are: refer to the red envelope generation mechanism of the WeChat platform to generate electronic red envelopes; or preset prize lists, and randomly generate prizes in the form of lottery, so as to effectively motivate heat users to actively participate in room temperature measurement, and facilitate the heating company to understand the room temperature data of heat users in order to adjust the heating supply; the present invention segments the room temperature image into image characters and measurement space positions, and extracts corresponding labels, constructs a convolutional neural network model to recognize the room temperature image, judges the validity of the measurement position and reads the corresponding room temperature value, and can effectively improve the reliability of room temperature measurement.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the systems, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a portion of code, and the module, program segment, or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0116] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0117] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0118] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A method for measuring room temperature of a thermal user based on temperature measurement images taken by a smart device, characterized in that: It includes: Step S1: The heat user obtains room temperature measurement values ​​at different room temperature measurement positions in real time through a room temperature measurement device within different preset time periods; Step S2: using an intelligent terminal device to collect images of multiple room temperature measurements and then sending them to a room temperature monitoring host computer through a communication module, and generating an incentive result according to a preset rule to encourage the user to measure the room temperature; Step S3: The room temperature monitoring host computer preprocesses, segments, and extracts labels on the room temperature image acquired in real time, and then inputs the image into the constructed convolutional neural network model for image recognition to determine the validity of the room temperature measurement position of the user on the image and identify the corresponding room temperature measurement value; The tag extraction in step S3 specifically includes the following steps: Taking the multiple segmented character images and spatial position images in the room temperature image as a set of images to be extracted, and extracting a convolution feature map of each image to be extracted by using a convolution layer in a convolutional neural network; In a feature mapping layer of a convolutional neural network, a plurality of extraction regions are set for each convolution feature map, and a region category and region information of each extraction region are used as respective region labels; the region categories include a room temperature measurement device region, a room temperature data region, and a room temperature measurement position region, and the region information includes at least digital character information and room temperature measurement position spatial information; Taking the region labels of all extracted regions corresponding to each convolution feature map as the label set of the convolution feature map, collecting the label sets of all convolution feature maps, and obtaining a room temperature label set, wherein the room temperature label set at least includes digital character information and room temperature measurement position spatial information; The images of the indoor measurement positions of the heat users are collected in advance and the corresponding tag identification points are extracted. The tag identification points exclude abnormal points such as windows, doorways and positions with cooling effects.

2. The method for measuring room temperature of a user based on temperature measurement images taken by a smart device according to claim 1, characterized in that: In step S1, the heat user obtains room temperature measurement values ​​at different room temperature measurement positions in real time through a room temperature measurement device within different preset time periods, including the following steps: For the heat users in the same community, different temperature measurement time periods are selected in the morning, afternoon and evening. The morning temperature measurement time period is set as the first time period t1, the noon temperature measurement time period is set as the second time period t2, and the evening temperature measurement time period is set as the third time period t3, expressed as t = [t1, t2, t3]; The thermal user selects at least one valid room temperature measurement location, represented as M = [M1, M2, ..., M i ], i≥1; The room temperature measurement device is used to measure the room temperature at different room temperature measurement positions in each time period to obtain the corresponding room temperature measurement values, which are expressed as and 3. The method for measuring room temperature of a thermal user based on temperature measurement images taken by a smart device according to claim 2, characterized in that: In step S2, the intelligent terminal device is used to respectively capture images of multiple room temperature measurement values ​​and transmits them to the room temperature monitoring host computer through the communication module, and generates an incentive result according to a preset rule to encourage the user to measure the room temperature, including the following steps: The intelligent terminal device uses its camera unit to measure the room temperature values ​​of different room temperature measurement positions within the first time period t1. Room temperature measurement values ​​at different room temperature measurement positions in the second time period t2 and the room temperature measurement values ​​at different room temperature measurement positions in the third time period t3 Shoot and obtain the original room temperature image data set, and send it to the room temperature monitoring host computer in sequence within the corresponding time period; Among them, when the thermal user uploads the corresponding room temperature image data in sequence within the preset first time period t1, the second time period t2 and the third time period t3, an incentive result is generated according to the preset rules to encourage the user.

4. The method for measuring room temperature of a thermal user based on temperature measurement images taken by a smart device according to claim 1, characterized in that: In step S3, the room temperature monitoring host computer preprocesses the room temperature image acquired in real time, including the following steps: Convert image signals into digital signals through acquisition and quantification; Image enhancement of room temperature images is performed through grayscale correction and grayscale transformation; The room temperature image is denoised and smoothed by local smoothing and bilateral filtering methods; The room temperature image dataset is subjected to image cropping, standardization and normalization in sequence; The calculation formula for the image standardization process is: μ is the mean of the image, x is the image matrix, σ is the standard deviation, and p is the number of pixels in the image.

5. The method for measuring room temperature of a user based on temperature measurement images taken by a smart device according to claim 1, characterized in that: The image segmentation in step S3 includes the following steps: After calculating the mean, variance, and grayscale value of the preprocessed input room temperature image, the optimal threshold for binary segmentation is calculated based on the mean and variance of the image. All pixels in the image with grayscale values ​​greater than the optimal threshold are marked as the target area, and all pixels in the image with grayscale values ​​less than the optimal threshold are marked as the background area. The calculation formula used is: Where, the background area is represented by 0, the target area is represented by 1, F(I,J) represents the input room temperature image, G(I,J) represents the output room temperature image, pixels are represented by i,j, the grayscale value of the input room temperature image is represented by f(i,j), the grayscale value of the output image is represented by g(i,j), and the optimal threshold is represented by T.

6. The method for measuring room temperature of a user based on temperature measurement images taken by a smart device according to claim 5, characterized in that: The image segmentation further includes segmenting the target area of ​​the image into image characters and spatial positions, which specifically includes the following steps: A deep learning image algorithm is used to build an image character recognition model and a spatial position model for identifying input room temperature images; The room temperature image is detected and located by using a detection algorithm to obtain the room temperature character area and the image non-character area, and the sizes of the sliding windows are set respectively; Sliding a sliding window from left to right within the room temperature character area to form a series of windows, and inputting the image area within the window into the image character recognition model to determine the probability of whether a single character is included in the window, and then optimizing the series of windows and their corresponding character probabilities, and the retained windows are the characters in the room temperature image; The sliding window is slid from left to right in the non-character area of ​​the image to form a series of windows, and the image area in the window is input into the spatial position model. It is determined whether the window includes multiple position nodes based on the pre-established two-dimensional image position node information library, and the retained window is the measurement space position in the room temperature image.

7. The method for measuring room temperature of a thermal user based on temperature measurement images taken by a smart device according to claim 1, characterized in that: In step S3, the construction of the convolutional neural network model specifically includes the following steps: During a historical time period, the image dataset processed by steps S1 and S2 is collected as sample data, the room temperature image set after preprocessing and image segmentation of the sample images is used as input variables, including a training set and a test set, and the room temperature label set is used as the output variable; Constructing a convolutional neural network, the convolutional neural network comprising an input layer, multiple convolutional layers and multiple pooling layers, multiple fully connected layers, and an output layer, setting a number of training times and a training target error, and training the convolutional neural network using room temperature images in the training set so that a feature value of the output layer of the convolutional neural network is the same as the label information of each input room temperature image; wherein the size of the input layer is the same as the predetermined size of the room temperature images in the training set; After the convolutional neural network training is completed, the convolutional neural network is tested using room temperature images in the test set. When the prediction error of the convolutional neural network in the test set is less than a preset value, the test is passed and the convolutional neural network can be used for room temperature measurement.

8. The method for measuring room temperature of a user based on temperature measurement images taken by a smart device according to claim 1, characterized in that: In step S3, the input is fed into the constructed convolutional neural network model for image recognition, determining the validity of the room temperature measurement position of the thermal user on the image and identifying the corresponding room temperature measurement value, specifically including: After the real-time user room temperature measurement image is subjected to the preprocessing, image segmentation and label extraction, it is input into the constructed convolutional neural network model to determine whether the room temperature measurement position of the hot user on the image is consistent with the preset room temperature target position, and then judge the validity of the spatial position. When the spatial position is valid, the corresponding room temperature measurement value is identified and read.

9. A system using the method for measuring room temperature of a thermal user based on taking temperature measurement images with a smart device according to any one of claims 1 to 8, characterized in that: It includes: The room temperature acquisition module is used for users to obtain room temperature measurement values ​​at different room temperature measurement positions in real time through the room temperature measurement device within different preset time periods; An image acquisition module is used to use an intelligent terminal device to respectively acquire images of multiple room temperature measurement values ​​and then send them to a room temperature monitoring host computer through a communication module; A user incentive module, used to generate incentive results according to preset rules to encourage users to measure room temperature; The image recognition module is used to preprocess, segment, and extract labels from the room temperature images acquired in real time by the room temperature monitoring host computer, and then input them into the constructed convolutional neural network model for image recognition, thereby determining the validity of the room temperature measurement position of the thermal user in the image and identifying the corresponding room temperature measurement value.

Images