A three-dimensional construction method of infrared images of resistor arrays based on neural network

Abstract

The present invention relates to the field of image processing technology, and in particular to a method for constructing a three-dimensional resistor array infrared scene based on a neural network, comprising: obtaining the overlap area of ​​a thermal value image according to the temperature performance and temperature index of the resistor, adjusting the pixel points of the overlap area, adjusting the pixel points of the buffer area between the overlap area and the non-overlap area, and completing the three-dimensional reconstruction of the continuous thermal value image. The present invention analyzes the temperature performance of the resistor in the continuous image, completes the matching of the overlap area of ​​the continuous two-dimensional image, and further combines the influence of the resistor on the background area and the influence between the resistors to complete the correction of the overlap area and the buffer area, so that the constructed three-dimensional model has better effect and higher quality.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for constructing three-dimensional resistor array infrared scenes based on a neural network. Background Art

[0002] Infrared imaging technology is widely used in security, industrial detection, medical diagnosis, military reconnaissance and other fields, and has become a relatively mature technology. Resistor array infrared scene generation is a key technology in semi-physical simulation experiments. By modeling the scene and target and analyzing the infrared characteristics, infrared images are generated using visible light image modulation.

[0003] When constructing the three-dimensional infrared scene of the resistor array, the three-dimensional image is mainly constructed by comparing the similar areas between the continuous infrared images. In the process of collecting the infrared images of the resistor array from multiple angles, the resistors in the resistor array should emit heat more stably. However, in the process of constructing the three-dimensional image from the two-dimensional image, there may be large temperature differences in some areas, and the image will show non-smooth characteristics, so the obtained three-dimensional image is not smooth enough. Summary of the invention

[0004] The invention provides a three-dimensional construction method of a resistor array infrared scene based on a neural network to solve the existing problems.

[0005] The three-dimensional construction method of the resistor array infrared scene based on neural network of the present invention adopts the following technical scheme:

[0006] An embodiment of the present invention provides a method for constructing a three-dimensional infrared scene of a resistor array based on a neural network, the method comprising the following steps:

[0007] Acquire several thermal images of the resistor array; divide each thermal image into a resistor area and a background area, acquire pixels of the resistor area and pixels of the background area, each resistor area corresponds to a resistor; establish a coordinate system, acquire the coordinates of any pixel point, and acquire the temperature value and grayscale value corresponding to each pixel point position;

[0008] Obtain the neighborhood of any resistor, fuse the temperature value of any resistor at each position in the neighborhood with the grayscale gradient expression of the pixel point, and obtain the temperature expression of any resistor in the neighborhood; obtain the adjacent resistors of any resistor, and obtain the temperature index of the resistor according to the temperature characteristics of any resistor and the temperature difference and distance relationship between the resistor and the adjacent resistors, combined with the temperature expression of the resistor in the neighborhood; match the consecutive thermal value images according to the similarity of the temperature index of the resistors between the consecutive thermal value images, and obtain the overlapping area of ​​any thermal value images;

[0009] According to the temperature difference and distance between any pixel point in the overlapped area of ​​the thermal value image and the adjacent resistor, the weight of the pixel point is obtained; according to the weight size relationship of the pixel point in the overlapped area of ​​the continuous thermal value images, the gray value of any pixel point in the overlapped area of ​​the thermal value image is adjusted to obtain the adjusted overlapped area of ​​all thermal value images;

[0010] According to the resistance distribution of the non-overlapping area, a buffer area of ​​the non-overlapping area of ​​any thermal value image is divided; based on the influence of the resistance of the overlapping area and the non-overlapping area on the temperature of the pixel points in the buffer area, the pixel points in the buffer area are adjusted, and the adjusted grayscale values ​​of the pixel points in the buffer area are obtained, and the adjusted buffer areas of all thermal value images are obtained;

[0011] The adjusted overlapped area and the adjusted buffer area of ​​any thermal value image are combined to perform three-dimensional reconstruction on the continuous thermal value image.

[0012] Furthermore, the method of obtaining the neighborhood of any resistor, fusing the temperature value of each position of the arbitrary resistor in the neighborhood with the grayscale gradient expression of the pixel point, and obtaining the temperature expression of the arbitrary resistor in the neighborhood includes the following specific methods:

[0013] The coordinates of the pixel points corresponding to the centroid of any resistor are marked as the position coordinates of the resistor. For any resistor, straight lines are drawn in several directions from the position coordinates of the resistor to obtain A pixel points closest to the position coordinates of the resistor in each direction as the neighborhood pixel points of the resistor. All the neighborhood pixel points of the resistor constitute the neighborhood of the resistor, and A is a preset first parameter.

[0014] The method for obtaining the temperature performance of any resistor in the neighborhood of any thermal value image is:

[0015]

[0016] in, Indicates The thermal image The temperature behavior of a resistor in a neighborhood is Indicates The thermal image The number of directions of the neighborhood pixels of a resistor; Indicates The thermal image The resistor is in the neighborhood The average temperature corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The sum of the temperatures corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient of the gray value of the pixel in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient direction of the grayscale value of the pixel in the direction is The angle between the directions.

[0017] Furthermore, the method of obtaining the adjacent resistors of any resistor, according to the temperature characteristics of any resistor and the temperature difference and distance relationship between the resistor and the adjacent resistors, combined with the temperature performance of the resistor in the neighborhood, obtaining the temperature index of the resistor includes the following specific methods:

[0018] Recording all resistors existing within a preset radius of the position coordinates of any resistor as neighboring resistors of the resistor;

[0019] Obtaining a temperature difference-distance ratio between any resistor and all adjacent resistors, and obtaining an average of the temperature difference-distance ratios between the resistor and all adjacent resistors, wherein the average is recorded as the difference between the resistor and the adjacent resistors;

[0020] The temperature of the resistor is the average temperature value of all pixel points in the resistor area to which the resistor belongs;

[0021] According to the temperature performance of any resistor in any thermal value image, the temperature of the resistor, and the difference between the resistor and adjacent resistors, the temperature index of the resistor in the thermal value image is obtained; the temperature performance of any resistor in any thermal value image, the temperature of the resistor, and the difference between the resistor and adjacent resistors are all proportional to the temperature index of the resistor in the thermal value image.

[0022] Furthermore, the method of matching the consecutive thermal value images according to the similarity of the temperature index of the resistor between the consecutive thermal value images to obtain the overlapping area of ​​any thermal value image includes the following specific methods:

[0023] From left to right, from top to bottom, The resistance of each thermal image is numbered. OK Column resistance;

[0024] The first The first column of the thermal image is in the order from left to right. The resistance of the thermal image is matched to obtain the The difference in the temperature index of each corresponding resistor in the image is less than or equal to the preset threshold 1 column resistor, the first The second column of the thermal image is The resistance of the next column of the column sequence resistance obtained in the image is matched, and so on, until the matched column sequence resistance no longer satisfies that the difference of the temperature index of each corresponding resistor is less than or equal to the preset threshold ;

[0025] The pixels of all the resistor areas that have been matched and the background area pixels in the same column as the pixels of all the resistor areas that have been matched together constitute the pixels of the overlapping area of ​​the thermal value image, and all the pixels of the overlapping area of ​​the thermal value image constitute the overlapping area of ​​the thermal value image.

[0026] Furthermore, the weight of the pixel is obtained according to the temperature difference and distance between any pixel in the overlapped area of ​​the thermal value image and the adjacent resistor, including the specific method of:

[0027] Record any pixel point in the background area of ​​the overlapping area on any thermal value image as a target pixel point, and record all resistors within a preset radius of the target pixel point as neighboring resistors of the target pixel point;

[0028] Get the influence of any adjacent resistor on the target pixel;

[0029] The average value of the influence of all adjacent resistors of the target pixel on the target pixel is obtained, and the average value is normalized and recorded as the weight of the pixel.

[0030] Furthermore, the specific method for obtaining the influence degree of any adjacent resistor on the target pixel point is:

[0031] The method for obtaining the influence of any adjacent resistor on the target pixel is:

[0032]

[0033] in, Indicates The influence of adjacent resistors on the target pixel point, Indicates The average temperature of adjacent resistors, Indicates the temperature of the target pixel position, Indicates the target pixel and the The distance between adjacent resistors, Indicates The area of ​​adjacent resistors, represents the absolute value function.

[0034] Furthermore, the grayscale value of any pixel in the overlapping area of ​​the thermal value images is adjusted by using the weight relationship of the pixel points in the overlapping area of ​​the continuous thermal value images to obtain the adjusted overlapping area of ​​all thermal value images, including the specific method of:

[0035] Compare the weights of any pixel in the overlapping area of ​​the thermal value image in two consecutive thermal value images, obtain the maximum value, and record it as the final weight of the pixel. The thermal value image to which the final weight belongs is recorded as the first thermal value image, and the other thermal value image is recorded as the second thermal value image. The weight of the pixel in the second thermal value image is recorded as ;

[0036] The gray value of the pixel point in the first thermal value image is multiplied by the final weight to obtain a first product; the gray value of the pixel point in the second thermal value image is multiplied by , obtaining a second product; adding the first product and the second product to obtain the adjusted grayscale value of the pixel;

[0037] The grayscale values ​​of all pixels in the overlapped area of ​​any thermal value image are adjusted to obtain the adjusted overlapped area of ​​all thermal value images.

[0038] Furthermore, the buffer area of ​​the non-overlapping area of ​​any thermal value image is divided according to the resistance distribution of the non-overlapping area, including the specific method of:

[0039] Preset distance , , obtain the distance between the pixel point of the resistor area closest to the overlap area and the overlap area, recorded as the first distance, when the first distance is greater than or equal to When the preset distance , non-overlapping areas extend by a preset distance The non-overlapping area within is set as the buffer area; when the first distance is greater than and less than When the preset distance , the distance between the first distance and the resistance area is greater than the preset distance The non-overlapping area is set as a buffer area; when the first distance is less than When , the distance between the pixel point farthest from the overlapping area and the overlapping area in the resistor closest to the overlapping area is obtained, which is recorded as the second distance, and the area within the second distance of the non-overlapping area close to the overlapping area is set as the buffer area.

[0040] Furthermore, the influence of the resistance of the combined overlapping area and the non-overlapping area on the temperature of the pixels in the buffer area is adjusted to obtain the adjusted grayscale value of the pixels in the buffer area, and the adjusted buffer area of ​​all thermal value images is obtained, including the specific method of:

[0041] The method for obtaining the grayscale value of any pixel in the background area of ​​the buffer area on the thermal image is as follows:

[0042]

[0043] in, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel is affected. Indicates The background area in the buffer area of ​​the thermal image The number of pixels belonging to the overlap area between the resistors closest to the overlap area in the horizontal direction, express The first pixel The adjusted gray value of each pixel is express The first pixel The distance between a pixel and the resistor closest to the overlap area, Indicates The background area in the buffer area of ​​the thermal image The distance from the pixel to the overlap area;

[0044] The method for obtaining the adjusted grayscale value of any pixel point in the background area of ​​the buffer area on the thermal value image is:

[0045]

[0046] in, Indicates The first thermal image of the background area in the buffer area The adjusted gray value of each pixel is Indicates The background area in the buffer area of ​​the thermal image The gray value of a pixel, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel;

[0047] The grayscale values ​​of all pixels in the buffer area of ​​any thermal value image are adjusted to obtain the adjusted buffer areas of all thermal value images.

[0048] Furthermore, the three-dimensional reconstruction of the continuous thermal value images by combining the adjusted overlapped area and the adjusted buffer area of ​​any thermal value image includes the following specific methods:

[0049] The pixels of the overlapping area of ​​any thermal value image and the background area in the buffer area are adjusted to obtain the adjusted thermal value image. The volume of all adjusted thermal value images is reconstructed using the Ray Casting algorithm. After obtaining preliminary three-dimensional volume data through the Ray Casting algorithm, the three-dimensional model is generated using the isosurface extraction technology.

[0050] The beneficial effects of the technical solution of the present invention are as follows: according to the characteristic that when a stable current flows through the resistors in the resistor matrix, the resistors emit stable heat, the temperature performance of the resistors in the continuous images is analyzed, and the overlapping area matching of the continuous two-dimensional images is completed. Furthermore, the influence of the resistors on the background area and the influence between the resistors are combined to complete the correction of the overlapping area and the non-overlapping area, so that the constructed three-dimensional model has better effect and higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flowchart of the steps of the method for constructing three-dimensional infrared scenes of a resistor array based on a neural network according to the present invention;

[0053] Figure 2 It is a schematic diagram of a buffer area for dividing non-overlapping areas of arbitrary thermal value images in the three-dimensional construction method of a resistor array infrared scene based on a neural network of the present invention. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of the three-dimensional construction method of the resistor array infrared scene based on the neural network proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0055] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0056] The specific scheme of the method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks provided by the present invention is described in detail below with reference to the accompanying drawings.

[0057] See also Figure 1 , which shows a flowchart of a method for constructing a three-dimensional infrared scene of a resistor array based on a neural network according to an embodiment of the present invention, the method comprising the following steps:

[0058] Step S001: obtain several thermal value images of the resistor array; divide each thermal value image into a resistor area and a background area, obtain the pixel points of the resistor area and the pixel points of the background area, each resistor area corresponds to a resistor; establish a coordinate system, obtain the coordinates of any pixel point, and obtain the temperature value and grayscale value corresponding to each pixel point position.

[0059] It should be noted that in the three-dimensional construction of the infrared scene of the resistor array, it is necessary to use multiple continuous two-dimensional thermal value images of the resistor array as a basis to help complete the construction of the three-dimensional image of the infrared scene of the resistor array, so it is necessary to collect thermal value images. The specific collection method is to use an infrared thermal imager to scan and collect multiple continuous two-dimensional thermal value images of the target resistor array from different angles.

[0060] Specifically, in order to implement the neural network-based resistor array infrared scene three-dimensional construction method proposed in this embodiment, it is first necessary to collect the thermal value image of the resistor array and process the thermal value image. The specific process is:

[0061] An infrared thermal imager is arranged in front of the resistor array, the resistor array is rotated at a preset speed, and the infrared thermal imager collects thermal value images of the resistor array at preset time intervals until the resistor array rotates 360 degrees, thereby obtaining several thermal value images of the resistor array.

[0062] It should be noted that, based on experience, the preset rotation speed is set to 60 seconds per revolution, and the preset time interval is set to 5 seconds, which can be adjusted according to actual conditions and are not specifically limited in this embodiment.

[0063] A convolutional neural network (CNN) with an Encoder-Decoder structure is used for several thermal value images of the resistor array to obtain several resistor areas in each thermal value image of the resistor array, and to obtain pixel points belonging to the resistor area and pixel points belonging to the background area.

[0064] Each resistance area is recorded as a resistor.

[0065] In the convolutional neural network CNN of the Encoder-Decoder structure, the pixel grayscale values ​​of the thermal image of the resistor array are used as the data set, and the stochastic gradient descent algorithm is used for training until convergence according to the cross entropy loss function. The labels are divided into two categories: resistor area and background area, and the pixels belonging to the resistor area and the background area in the thermal image of the resistor array can be classified.

[0066] It should be noted that the convolutional neural network CNN of the Encoder-Decoder structure is an existing model structure for image pixel-level semantic segmentation, so it is not described in detail in this embodiment.

[0067] The first pixel point in the lower left corner of each thermal value image is taken as the origin of the coordinate system. The first pixel point in the lower left corner is horizontally to the right as the positive direction of the X-axis, and the first pixel point in the lower left corner is vertically upward as the positive direction of the Y-axis. The coordinate system is constructed. The horizontal distance and vertical distance between any pixel point and the origin constitute the X-axis and Y-axis coordinates of the pixel point.

[0068] So far, several thermal value images of the resistor array and the coordinates of the pixel points of the resistor area and the pixel points of the background area in each thermal value image are obtained through the above method.

[0069] Step S002: Acquire the neighborhood of any resistor, fuse the temperature value of any resistor at each position in the neighborhood and the grayscale gradient expression of the pixel point to obtain the temperature expression of any resistor in the neighborhood; acquire the adjacent resistors of any resistor, and acquire the temperature index of the resistor according to the temperature characteristics of any resistor and the temperature difference and distance relationship between the resistor and the adjacent resistors, combined with the temperature expression of the resistor in the neighborhood; match the consecutive thermal value images according to the similarity of the temperature index of the resistors between the consecutive thermal value images to acquire the overlapping area of ​​any thermal value images.

[0070] It should be noted that when constructing a three-dimensional image, continuous two-dimensional images are usually combined and reconstructed according to certain rules to form a three-dimensional image. For a resistor array image, when the current flowing through the resistor is stable, the heat generated by the resistor tends to be stable. According to this rule, the temperature performance of each resistor can be determined, and the overlapping part of the continuous thermal value image can be obtained according to the similarity of the temperature performance of the resistor on the continuous image.

[0071] It should be noted that, since the arrangement and distribution of resistors in the resistor array has obvious regularity, it is impossible to directly determine which part of the resistors in the captured image belong to. However, due to the different temperature performance of the resistors themselves and the different degrees of mutual influence between resistors, there are certain differences in the temperature performance in the neighborhoods of different resistors. Therefore, by analyzing the temperature performance of the resistors themselves and the temperature performance in the neighborhood of the resistors, the resistance matching of adjacent images can be completed, thereby determining the overlapping area of ​​adjacent images.

[0072] Specifically, in step 2.1, the neighborhood of any resistor is obtained, and the temperature value of each position of the arbitrary resistor in the neighborhood and the grayscale gradient expression of the pixel point are merged to obtain the temperature expression of the arbitrary resistor in the neighborhood.

[0073] It should be noted that since the temperature around the resistor and the grayscale value changes of the pixels around the resistor can both show the temperature characteristics of the resistor, it is necessary to obtain the temperature of the resistor neighborhood and the grayscale value changes of the pixels. First, the exact range of the neighborhood of any resistor needs to be obtained. The thermal value image can be coordinate-based, and then the neighborhood of the resistor can be obtained based on the coordinate distance of the pixel points and the coordinate of the resistor.

[0074] First, the coordinates of the pixel points corresponding to the centroid of any resistor are marked as the position coordinates of the resistor. For any resistor, straight lines are drawn in several directions with the position coordinates of the resistor to obtain A pixel points closest to the position coordinates of the resistor in each direction as the neighborhood pixel points of the resistor. All the neighborhood pixel points of the resistor constitute the neighborhood of the resistor, and A is a preset first parameter.

[0075] It should be noted that the several directions include up, down, left, right, upper left, lower left, upper right, and lower right, which are eight directions in total.

[0076] It should be noted that A is preset to 20 based on experience and can be adjusted according to actual conditions, and is not specifically limited in this embodiment.

[0077] It should be noted that after obtaining the neighborhood of the resistor, the temperature performance of the resistor in the neighborhood can be obtained based on the temperature of the resistor in the neighborhood and the grayscale change of the pixel point.

[0078] It should be noted that for any resistor, within the neighborhood, the higher the temperature, the larger the influence range of the resistor, and the stronger the temperature performance of the resistor within the neighborhood; at the same time, the larger the grayscale gradient in each direction within the neighborhood, the more drastic the temperature change, and the more sensitive the resistor is to the temperature, so the stronger the temperature performance of the resistor within the neighborhood.

[0079] Therefore, the temperature performance of the resistor in the neighborhood can be calculated based on the temperature characteristics and grayscale changes of the resistor in the neighborhood.

[0080] As an embodiment, a method for obtaining the temperature performance of any resistor in a neighborhood in any thermal value image is as follows:

[0081]

[0082] in, Indicates The thermal image The temperature behavior of a resistor in a neighborhood is Indicates The thermal image The number of directions of the neighborhood pixels of a resistor; Indicates The thermal image The resistor is in the neighborhood The average temperature corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The sum of the temperatures corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient of the gray value of the pixel in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient direction of the grayscale value of the pixel in the direction is The angle between the directions.

[0083] It should be noted that the temperature performance of the resistor in the neighborhood is the specific performance of the temperature values ​​corresponding to the neighborhood pixels in each direction in the resistor neighborhood. , combined with the gradient performance of the resistance at the pixel points in all directions Since the temperature performance of the resistor neighborhood pixel is mainly affected by the resistor, different resistors have different degrees of influence on the positions of the neighborhood pixel due to their own temperature differences and temperature magnitudes, resulting in different temperature performances corresponding to the neighborhood pixel points. Therefore, it can represent the temperature performance of the resistor in the neighborhood.

[0084] Step 2.2, obtaining the neighboring resistors of any resistor, and obtaining the temperature index of the resistor according to the temperature characteristics of the arbitrary resistor and the temperature difference and distance relationship between the resistor and the neighboring resistors, combined with the temperature performance of the resistor in the neighborhood.

[0085] It should be noted that the temperature index of a resistor is composed of three parts: the temperature corresponding to several pixels in the resistor area, the temperature performance of the resistor in the neighborhood, and the correlation between the resistor and the adjacent resistors.

[0086] It should be noted that the higher the temperature of the resistance area, the stronger the temperature performance of the resistance in the neighborhood, and the greater the correlation between the resistance and the adjacent resistance, the higher the temperature index of the resistance.

[0087] First, all resistors existing within a preset radius of the position coordinates of any resistor are recorded as neighboring resistors of the resistor.

[0088] It should be noted that the preset radius is set to the length of the thermal value image based on experience. , can be adjusted according to actual conditions, and this embodiment does not make any specific limitation.

[0089] Secondly, according to the distance difference and temperature difference between any resistor and adjacent resistors, the difference between the resistor and adjacent resistors is obtained.

[0090] It should be noted that the correlation between any resistor and adjacent resistors is manifested in: the temperature difference-distance ratio of any resistor to all adjacent resistors. The larger the temperature difference-distance ratio of the resistor to all adjacent resistors, the greater the temperature difference and the closer the distance. Then, the stronger the temperature performance of the resistor, the greater the temperature index of the resistor.

[0091] As an embodiment, a method for obtaining the difference between any resistor and adjacent resistors is as follows:

[0092] The average temperature corresponding to all pixel points of any resistor is recorded as the temperature of the resistor.

[0093]

[0094] in, Indicates The thermal image The difference between a resistor and adjacent resistors; Indicates The thermal image The number of adjacent resistors of a resistor, Indicates The thermal image The resistor The adjacent resistor The temperature difference of the resistor, Indicates The thermal image The resistor The adjacent resistor The coordinate distance of the resistor.

[0095] It should be noted that Indicates The first thermal image The temperature difference-distance ratio of the resistor to the zth adjacent resistor. The size of this value reflects the difference between the resistor and the adjacent resistors. If the temperature difference is large and the distance is small, it means that the difference between the resistor and the adjacent resistors is large. If the difference between the resistor and all adjacent resistors is large, the temperature performance of the resistor is more prominent and the temperature index is larger.

[0096] Finally, the temperature index of the resistor is obtained comprehensively based on the temperature of the resistor, the difference between the resistor and adjacent resistors, and the temperature performance of the resistor in the neighborhood.

[0097] As an embodiment, a method for obtaining a temperature index of any resistor in any thermal value image is as follows:

[0098]

[0099] in, Indicates The thermal image The temperature index of a resistor, Indicates The thermal image The temperature of a resistor, Indicates The thermal image The temperature behavior of a resistor in a neighborhood is Indicates The thermal image The difference between a resistor and adjacent resistors, Represents the normalization function.

[0100] It should be noted that the higher the temperature of the resistor area, the stronger the temperature performance of the resistor in the neighborhood, and the greater the difference between the resistor and the adjacent resistors, the higher the temperature index of the resistor.

[0101] Step 2.3, according to the similarity of the temperature index of the resistor between the consecutive thermal value images, the consecutive thermal value images are matched to obtain the overlapping area of ​​any thermal value image.

[0102] It should be noted that for two consecutive thermal value images, the overlapping area shows that the temperature index of each corresponding resistor in the area is similar. Therefore, the overlapping area of ​​the consecutive thermal value images can be obtained according to the similarity of the temperature index of the resistors in some areas of the consecutive thermal value images.

[0103] It should be noted that, for two consecutive thermal value images, because the only difference is the angle of the resistor array when shooting, the row order of the resistors in the resistor array is also the row order of the resistors in the thermal value image, and the row order of any resistor in the resistor array in different thermal value images is consistent. Therefore, the matching of consecutive thermal value images can be achieved by placing the consecutive thermal value images side by side, only moving the latter thermal value image to overlap the two thermal value images, and judging whether the overlapping area is the overlapping area of ​​the consecutive thermal value images based on the matching of the resistors in the overlapping area.

[0104] From left to right, from top to bottom, The resistance of each thermal image is numbered. OK Column resistance.

[0105] The first The first column of the thermal image is in the order from left to right. The resistance of the thermal image is matched to obtain the The difference in the temperature index of each corresponding resistor in the image is less than or equal to the preset threshold 1 column resistor, the first The second column of the thermal image is The resistance of the next column of the column sequence resistance obtained in the image is matched, and so on, until the matched column sequence resistance no longer satisfies that the difference of the temperature index of each corresponding resistor is less than or equal to the preset threshold The pixels of all the resistor regions that have been matched and the pixels of the background region that are in the same column as the pixels of all the resistor regions that have been matched together constitute the pixels of the overlapped region of the thermal value image, and all the pixels of the overlapped region of the thermal value image constitute the overlapped region of the thermal value image.

[0106] It should be noted that the threshold value D is preset to 0.5 based on experience and can be adjusted according to actual conditions, and is not specifically limited in this embodiment.

[0107] So far, the overlapping area of ​​any thermal value image is obtained through the above method.

[0108] Step S003: Obtain the weight of any pixel point in the overlapping area of ​​the thermal value image according to the temperature difference and distance between the pixel point and the adjacent resistor; adjust the grayscale value of any pixel point in the overlapping area of ​​the thermal value image according to the weight relationship of the pixel point in the overlapping area of ​​the continuous thermal value images, and obtain the adjusted overlapping area of ​​all thermal value images.

[0109] It should be noted that for thermal value images, due to the heat generated by resistors, the three-dimensional construction between continuous thermal value images cannot simply use the overlapping area of ​​a certain image as the connection area. It is necessary to analyze it in combination with the heat generated by resistors. The closer the part is to the resistor area, the higher the temperature should be. Therefore, it is necessary to calculate the weight of the pixels in the overlapping area on the continuous thermal value images. And the buffer area between the overlapping area and the non-overlapping area should also be adjusted to make the image connection smoother, further improving the construction quality of the three-dimensional image.

[0110] It should be noted that, since the current passing through the resistor is stable, only the pixels in the background area are adjusted.

[0111] Specifically, in step 3.1, the weight of any pixel point is obtained according to the temperature difference and distance between any pixel point in the overlapped area of ​​the thermal value image and the adjacent resistor.

[0112] It should be noted that in the thermal value image, the temperature performance of the background area is affected by the heat generation of the resistor area, and the number and position of the resistors will affect the temperature performance of the background area. Therefore, the weight of the pixel points in the background area should consider the position of the resistors in the neighborhood of the pixel points and the corresponding temperature.

[0113] First, any pixel point in the background area in the overlapped area on any thermal value image is recorded as a target pixel point, and all resistors within a preset radius of the target pixel point are recorded as neighboring resistors of the target pixel point.

[0114] It should be noted that the preset radius is set to the length of the thermal value image based on experience. , can be adjusted according to actual conditions, and this embodiment does not make any specific limitation.

[0115] Secondly, according to the distance between any target pixel and any adjacent resistor and the temperature performance of the adjacent resistor, the influence degree of the adjacent resistor on the target pixel is obtained.

[0116] As an embodiment, a method for obtaining the influence degree of any adjacent resistor on the target pixel point is:

[0117]

[0118] in, Indicates The influence of adjacent resistors on the target pixel point, Indicates The average temperature of adjacent resistors, Indicates the temperature of the target pixel position, Indicates the target pixel and the The distance between adjacent resistors, Indicates The area of ​​adjacent resistors, represents the absolute value function.

[0119] It should be noted that for resistors that affect pixel points, the closer they are to the pixel points and the larger their area, the greater their influence. Since the heating degree of the resistors does not change when the current is stable in the resistor array, we calculate the weight according to the influence of each resistor on this point. The greater the influence, the greater the weight.

[0120] Next, the average value of the influence of all the adjacent resistors of the target pixel on the target pixel is obtained, and the average value is normalized and recorded as the weight of the pixel.

[0121] Step 3.2, using the weight relationship of the pixels in the overlapping area of ​​the continuous thermal value images, adjust the grayscale value of any pixel in the overlapping area of ​​the thermal value images to obtain the adjusted overlapping area of ​​all thermal value images.

[0122] It should be noted that in the overlapping areas of continuous thermal value images, the different weights of corresponding pixels can reflect the importance of the information of the pixel points in different thermal value images. In the process of constructing a three-dimensional image, the pixel points of the thermal value image with greater importance should have greater weights. The pixel points of the background area in the overlapping area on the thermal value image are adjusted using the pixel weights to obtain the adjusted pixel grayscale values, so that the three-dimensional image construction effect is better.

[0123] Compare the weights of any pixel in the overlapping area of ​​the thermal value image in two consecutive thermal value images, obtain the maximum value, and record it as the final weight of the pixel. The thermal value image to which the final weight belongs is recorded as the first thermal value image, and the other thermal value image is recorded as the second thermal value image. The weight of the pixel in the second thermal value image is recorded as ;

[0124] The gray value of the pixel point in the first thermal value image is multiplied by the final weight to obtain a first product; the gray value of the pixel point in the second thermal value image is multiplied by , obtain a second product; add the first product and the second product to obtain the adjusted grayscale value of the pixel.

[0125] It should be noted that, since the color in the thermal value image reflects the temperature, the weight calculated using the temperature can be used to adjust the grayscale value of the pixel.

[0126] The grayscale values ​​of all pixels in the overlapped area of ​​any thermal value image are adjusted to obtain the adjusted overlapped area of ​​all thermal value images.

[0127] So far, the adjusted overlapped area of ​​any thermal value image is obtained through the above method.

[0128] Step S004: Divide the buffer area of ​​the non-overlapping area of ​​any thermal value image according to the resistance distribution of the non-overlapping area; adjust the pixel points of the buffer area based on the influence of the resistance of the overlapping area and the non-overlapping area on the temperature of the pixel points in the buffer area, obtain the adjusted grayscale value of the pixel points in the buffer area, and obtain the adjusted buffer area of ​​all thermal value images.

[0129] It should be noted that in order to make the overlapping area and the non-overlapping area smoother after the continuous thermal value images are connected, it is necessary to divide the part where the overlapping area and the non-overlapping area are connected into a buffer area and process the buffer area. Similarly, there will be a resistor area and a background area on the non-overlapping area. Since the current passing through the resistor is stable, only the pixels in the background area are adjusted.

[0130] Specifically, in step 4.1, a buffer area of ​​the non-overlapping area of ​​any thermal value image is divided according to the resistance distribution of the non-overlapping area.

[0131] It should be noted that in order to retain the characteristic performance of the non-overlapping area as much as possible, the buffer area should not be set too large, and in order to ensure smooth connection, the buffer area should not be too small. Because the characteristics of the thermal value image of the resistor array are mainly manifested in the resistor area, the buffer area of ​​each thermal value image can be specifically obtained based on the resistance performance of the non-overlapping area close to the overlapping area.

[0132] It should be noted that if the distance between the resistor closest to the overlapping area and the overlapping area is too large, the resistor closest to the overlapping area has little effect on the buffer area, and there is no need to consider the effect of the resistor closest to the overlapping area. The area within a preset distance from the overlapping area can be recorded as the buffer area; if the resistor closest to the overlapping area has an effect on the buffer area, the background area in the neighborhood of the resistor closest to the overlapping area is retained, and the remaining part close to the overlapping area is recorded as the buffer area; if the resistor closest to the overlapping area has a greater effect on the buffer area, all areas between the resistor area closest to the overlapping area and the overlapping area are recorded as the buffer area.

[0133] Preset distance , , obtain the distance between the pixel point of the resistor area closest to the overlap area and the overlap area, recorded as the first distance, when the first distance is greater than or equal to When the preset distance , non-overlapping areas extend by a preset distance The non-overlapping area within is set as the buffer area; when the first distance is greater than and less than When the preset distance , the distance between the first distance and the resistance area is greater than the preset distance The non-overlapping area is set as a buffer area; when the first distance is less than When , the distance between the pixel point farthest from the overlapping area and the overlapping area in the resistor closest to the overlapping area is obtained, which is recorded as the second distance, and the area within the second distance of the non-overlapping area close to the overlapping area is set as the buffer area.

[0134] See the detailed schematic diagram for details. Figure 2 , where R1, R2, R3, R4, R5, and R6 are resistors forming a resistor array, where the left side of L is the overlapping area and the right side is the non-overlapping area.

[0135] It should be noted that , , , , All are preset distance values. for , preset for , preset 20 pixels, preset for , can be adjusted according to actual conditions, and this embodiment does not make any specific limitation.

[0136] Step 4.2, combining the influence of the resistance of the overlapping area and the non-overlapping area on the temperature of the pixels in the buffer area, adjusting the pixels in the buffer area, obtaining the adjusted grayscale values ​​of the pixels in the buffer area, and obtaining the adjusted buffer areas of all thermal value images.

[0137] It should be noted that the adjustment direction of the pixel points in the background area in the buffer area is related to the heat and distance of the surrounding resistors. Therefore, the affected grayscale value of the pixel points in the background area in the buffer area can be obtained first according to the position of the pixel points in the background area in the buffer area and the ratio of the grayscale value of the pixel points in the horizontal direction to the distance.

[0138] As an embodiment, a method for obtaining the grayscale value of any pixel in the background area in the buffer area on the thermal image is as follows:

[0139]

[0140] in, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel is affected. Indicates The background area in the buffer area of ​​the thermal image The number of pixels belonging to the overlap area between the resistors closest to the overlap area in the horizontal direction, express The first pixel The adjusted gray value of each pixel is express The first pixel The distance between a pixel and the resistor closest to the overlap area, Indicates The background area in the buffer area of ​​the thermal image The distance from the pixel to the overlap area.

[0141] Next, the adjusted grayscale value of the pixel is obtained based on the influence grayscale value of any pixel in the background area in the buffer area on the thermal value image and the grayscale value of the pixel.

[0142] As an embodiment, a method for obtaining the adjusted grayscale value of any pixel point in the background area of ​​the buffer area on the thermal value image is as follows:

[0143]

[0144] in, Indicates The first thermal image of the background area in the buffer area The adjusted gray value of each pixel is Indicates The background area in the buffer area of ​​the thermal image The gray value of a pixel, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel is affected.

[0145] It should be noted that the relationship between the grayscale value and the distance of the pixel points in the buffer area and the resistor in the overlapping area in the horizontal direction is utilized. Since the heat generated by the resistor will be affected by the distance during the transfer process, the ratio of the grayscale value of the pixel point and the distance in the horizontal direction is calculated based on the pixel position, and then combined with the original grayscale value, the average grayscale value is calculated as the corrected grayscale value.

[0146] The grayscale values ​​of all pixels in the buffer area of ​​any thermal value image are adjusted to obtain the adjusted buffer areas of all thermal value images.

[0147] So far, the adjusted buffer area of ​​any thermal value image is obtained through the above method.

[0148] Step S005: Combining the adjusted overlapped area and the adjusted buffer area of ​​any thermal value image, three-dimensionally reconstruct the continuous thermal value image.

[0149] The pixels of the overlapping area of ​​any thermal value image and the background area in the buffer area are adjusted to obtain the adjusted thermal value image. The volume of all the adjusted thermal value images is reconstructed using the Ray Casting algorithm. After the preliminary three-dimensional volume data is obtained by the Ray Casting algorithm, the three-dimensional model is generated using the Iso-surfaces Extraction Technique.

[0150] It should be noted that the Ray Casting algorithm is an existing direct volume rendering algorithm based on image sequences, so it will not be described in detail in this embodiment.

[0151] It should be noted that the isosurface extraction technology is an existing method for visualizing three-dimensional spatial data fields, so it is not described in detail in this embodiment.

[0152] At this point, the three-dimensional construction of the resistor array infrared scene is completed through the above method.

[0153] At this point, this embodiment is completed.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional construction method of a resistor array infrared scene based on a neural network, characterized in that: The method comprises the following steps: Acquire several thermal images of the resistor array; divide each thermal image into a resistor area and a background area, acquire pixels of the resistor area and pixels of the background area, each resistor area corresponds to a resistor; establish a coordinate system, acquire the coordinates of any pixel point, and acquire the temperature value and grayscale value corresponding to each pixel point position; Obtain the neighborhood of any resistor, fuse the temperature value of any resistor at each position in the neighborhood with the grayscale gradient expression of the pixel point, and obtain the temperature expression of any resistor in the neighborhood; obtain the adjacent resistors of any resistor, and obtain the temperature index of the resistor according to the temperature characteristics of any resistor and the temperature difference and distance relationship between the resistor and the adjacent resistors, combined with the temperature expression of the resistor in the neighborhood; match the consecutive thermal value images according to the similarity of the temperature index of the resistors between the consecutive thermal value images, and obtain the overlapping area of ​​any thermal value images; According to the temperature difference and distance between any pixel point in the overlapped area of ​​the thermal value image and the adjacent resistor, the weight of the pixel point is obtained; according to the weight size relationship of the pixel point in the overlapped area of ​​the continuous thermal value images, the gray value of any pixel point in the overlapped area of ​​the thermal value image is adjusted to obtain the adjusted overlapped area of ​​all thermal value images; According to the resistance distribution of the non-overlapping area, a buffer area of ​​the non-overlapping area of ​​any thermal value image is divided; based on the influence of the resistance of the overlapping area and the non-overlapping area on the temperature of the pixel points in the buffer area, the pixel points in the buffer area are adjusted, and the adjusted grayscale values ​​of the pixel points in the buffer area are obtained, and the adjusted buffer areas of all thermal value images are obtained; The adjusted overlapped area and the adjusted buffer area of ​​any thermal value image are combined to perform three-dimensional reconstruction on the continuous thermal value image.

2. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1 is characterized in that: The method of obtaining the neighborhood of any resistor, fusing the temperature value of each position of the arbitrary resistor in the neighborhood with the grayscale gradient expression of the pixel point, and obtaining the temperature expression of the arbitrary resistor in the neighborhood includes the following specific methods: The coordinates of the pixel points corresponding to the centroid of any resistor are marked as the position coordinates of the resistor. For any resistor, straight lines are drawn in several directions from the position coordinates of the resistor to obtain A pixel points closest to the position coordinates of the resistor in each direction as the neighborhood pixel points of the resistor. All the neighborhood pixel points of the resistor constitute the neighborhood of the resistor, and A is a preset first parameter. The method for obtaining the temperature performance of any resistor in the neighborhood of any thermal value image is: in, Indicates The thermal image The temperature behavior of a resistor in a neighborhood is Indicates The thermal image The number of directions of the neighborhood pixels of a resistor; Indicates The thermal image The resistor is in the neighborhood The average temperature corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The sum of the temperatures corresponding to the neighboring pixels in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient of the gray value of the pixel in each direction; Indicates The thermal image The resistor is in the neighborhood The gradient direction of the grayscale value of the pixel in the direction is The angle between the directions.

3. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1, characterized in that: The method of obtaining the neighboring resistors of any resistor, according to the temperature characteristics of any resistor and the temperature difference and distance relationship between the resistor and the neighboring resistors, and in combination with the temperature performance of the resistor in the neighborhood, obtaining the temperature index of the resistor includes the following specific methods: Recording all resistors existing within a preset radius of the position coordinates of any resistor as neighboring resistors of the resistor; Obtaining a temperature difference-distance ratio between any resistor and all adjacent resistors, and obtaining an average of the temperature difference-distance ratios between the resistor and all adjacent resistors, wherein the average is recorded as the difference between the resistor and the adjacent resistors; The temperature of the resistor is the average temperature value of all pixel points in the resistor area to which the resistor belongs; According to the temperature performance of any resistor in any thermal value image, the temperature of the resistor, and the difference between the resistor and adjacent resistors, the temperature index of the resistor in the thermal value image is obtained; the temperature performance of any resistor in any thermal value image, the temperature of the resistor, and the difference between the resistor and adjacent resistors are all proportional to the temperature index of the resistor in the thermal value image.

4. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1, characterized in that: The method of matching the continuous thermal value images according to the similarity of the temperature index of the resistor between the continuous thermal value images and obtaining the overlapped area of ​​any thermal value image includes the following specific methods: From left to right, from top to bottom, The resistance of each thermal image is numbered. OK Column resistance; The first The first column of the thermal image is in the order from left to right. The resistance of the thermal image is matched to obtain the The difference in the temperature index of each corresponding resistor in the image is less than or equal to the preset threshold 1 column resistor, the first The second column of the thermal image is The resistance of the next column of the column sequence resistance obtained in the image is matched, and so on, until the matched column sequence resistance no longer satisfies that the difference of the temperature index of each corresponding resistor is less than or equal to the preset threshold ; The pixels of all the resistor areas that have been matched and the background area pixels in the same column as the pixels of all the resistor areas that have been matched together constitute the pixels of the overlapping area of ​​the thermal value image, and all the pixels of the overlapping area of ​​the thermal value image constitute the overlapping area of ​​the thermal value image.

5. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1 is characterized in that: The specific method of obtaining the weight of the pixel point according to the temperature difference and distance between any pixel point and the adjacent resistor in the overlapped area of ​​the thermal value image includes: Record any pixel point in the background area of ​​the overlapping area on any thermal value image as a target pixel point, and record all resistors within a preset radius of the target pixel point as neighboring resistors of the target pixel point; Get the influence of any adjacent resistor on the target pixel; The average value of the influence of all the adjacent resistors of the target pixel on the target pixel is obtained, and the average value is normalized and recorded as the weight of the pixel.

6. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 5 is characterized in that: The specific method for obtaining the influence degree of any adjacent resistor on the target pixel point is: The method for obtaining the influence of any adjacent resistor on the target pixel is: in, Indicates The influence of adjacent resistors on the target pixel point, Indicates The average temperature of adjacent resistors, Indicates the temperature of the target pixel position, Indicates the target pixel and the The distance between adjacent resistors, Indicates The area of ​​adjacent resistors, represents the absolute value function.

7. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1, characterized in that: The method of adjusting the grayscale value of any pixel in the overlapping area of ​​the thermal value images by using the weight relationship of the pixel points in the overlapping area of ​​the continuous thermal value images to obtain the adjusted overlapping area of ​​all thermal value images includes the following specific methods: Compare the weights of any pixel in the overlapping area of ​​the thermal value image in two consecutive thermal value images, obtain the maximum value, and record it as the final weight of the pixel. The thermal value image to which the final weight belongs is recorded as the first thermal value image, and the other thermal value image is recorded as the second thermal value image. The weight of the pixel in the second thermal value image is recorded as ; The gray value of the pixel point in the first thermal value image is multiplied by the final weight to obtain a first product; the gray value of the pixel point in the second thermal value image is multiplied by , obtaining a second product; adding the first product and the second product to obtain the adjusted grayscale value of the pixel; The grayscale values ​​of all pixels in the overlapped area of ​​any thermal value image are adjusted to obtain the adjusted overlapped area of ​​all thermal value images.

8. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1 is characterized in that: The method of dividing the buffer area of ​​the non-overlapping area of ​​any thermal value image according to the resistance distribution of the non-overlapping area includes: Preset distance , , obtain the distance between the pixel point of the resistor area closest to the overlap area and the overlap area, recorded as the first distance, when the first distance is greater than or equal to When the preset distance , non-overlapping areas extend by a preset distance The non-overlapping area within is set as the buffer area; when the first distance is greater than and less than When the preset distance , the distance between the first distance and the resistance area is greater than the preset distance The non-overlapping area is set as a buffer area; when the first distance is less than When , the distance between the pixel point farthest from the overlapping area and the overlapping area in the resistor closest to the overlapping area is obtained, which is recorded as the second distance, and the area within the second distance of the non-overlapping area close to the overlapping area is set as the buffer area.

9. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1, characterized in that: The method of combining the influence of the resistance of the overlap area and the non-overlap area on the temperature of the pixel points in the buffer area, adjusting the pixel points in the buffer area, obtaining the adjusted grayscale value of the pixel points in the buffer area, and obtaining the adjusted buffer area of ​​all thermal value images includes: The method for obtaining the grayscale value of any pixel in the background area of ​​the buffer area on the thermal image is as follows: in, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel is affected. Indicates The background area in the buffer area of ​​the thermal image The number of pixels belonging to the overlap area between the resistors closest to the overlap area in the horizontal direction, express The first pixel The adjusted gray value of each pixel is express The first pixel The distance between a pixel and the resistor closest to the overlap area, Indicates The background area in the buffer area of ​​the thermal image The distance from the pixel to the overlap area; The method for obtaining the adjusted grayscale value of any pixel point in the background area of ​​the buffer area on the thermal value image is: in, Indicates The first thermal image of the background area in the buffer area The adjusted gray value of each pixel is Indicates The background area in the buffer area of ​​the thermal image The gray value of a pixel, Indicates The background area in the buffer area of ​​the thermal image The gray value of each pixel; The grayscale values ​​of all pixels in the buffer area of ​​any thermal value image are adjusted to obtain the adjusted buffer areas of all thermal value images.

10. The method for constructing three-dimensional infrared scenes of resistor arrays based on neural networks according to claim 1, characterized in that: The specific method of combining the adjusted overlapped area and the adjusted buffer area of ​​any thermal value image to perform three-dimensional reconstruction on the continuous thermal value image includes: The pixels of the overlapping area of ​​any thermal value image and the background area in the buffer area are adjusted to obtain the adjusted thermal value image. The volume of all adjusted thermal value images is reconstructed using the Ray Casting algorithm. After obtaining preliminary three-dimensional volume data through the Ray Casting algorithm, the three-dimensional model is generated using the isosurface extraction technology.

Images