Infrared image encoding method, device and equipment for pulse neural network

Abstract

Embodiments of the present disclosure disclose an infrared image encoding method, device and equipment for a pulse neural network. A specific implementation of the method comprises: determining a gray scale range and a mapping range according to a histogram of an original infrared image; determining at least one enhancement range to be enhanced in the gray scale range; generating corresponding enhancement parameters; constructing a hybrid mapping function according to the gray scale range, the mapping range and the obtained enhancement parameter set; inputting each pixel value in the original infrared image into the hybrid mapping function to obtain a mapping image; converting each mapping pixel value in the mapping image into a pulse sequence through a Bernoulli experiment; splicing each pulse sequence as an infrared image encoding result; and inputting the infrared image encoding result into a pulse neural network for training. The implementation can reduce the time of a pseudo-color conversion link in a computer recognition task, and improve the computing speed and related performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an infrared image encoding method, device and equipment for pulse neural network. BACKGROUND

[0002] The picture taken by an infrared camera is usually presented in the form of a thermal image, and the human eye cannot see infrared light, so it is necessary to convert the picture taken by the infrared camera into a visual image for pseudo-color mapping. For infrared image encoding, the commonly used method is to perform pseudo-color mapping to map infrared signals of different intensities into the visible color range.

[0003] However, when the above method is used for infrared image encoding, the following technical problems often exist:

[0004] First, when manually annotating an infrared image dataset, the invisible infrared signal intensity needs to be pseudo-color coded and mapped through the RGB / HSV visible light channel. The pseudo-color coding and mapping may mask the original grayscale features, exacerbate noise interference in model training, and cause precision to decline.

[0005] Second, real-time pseudo-color conversion and signal processing in the computer deployment stage will occupy a large amount of computing resources, causing algorithm redundancy and energy waste.

[0006] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present inventive concepts, and therefore, it can contain information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0007] The summary section of the present disclosure is used to introduce the concepts in a brief form, which will be described in detail in the specific embodiments section. The summary section of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to be used to limit the scope of the claimed technical solutions.

[0008] Some embodiments of the present disclosure propose an infrared image encoding method, device and equipment for pulse neural network to solve one or more of the technical problems mentioned in the above BACKGROUND section.

[0009] In a first aspect, some embodiments of the present disclosure provide an infrared image encoding method for a spiking neural network, comprising: determining a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt an input of the spiking neural network; determining at least one enhancement range to be enhanced in the gray scale range; generating an enhancement parameter corresponding to each of the at least one enhancement range; constructing a hybrid mapping function according to the gray scale range, the mapping range and the obtained set of enhancement parameters; inputting each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, thereby obtaining a mapped image; converting each mapped pixel value in the mapped image into a pulse sequence through a Bernoulli experiment, thereby obtaining a set of pulse sequences; splicing each pulse sequence in the set of pulse sequences to generate three-dimensional data, thereby obtaining a set of three-dimensional data as an infrared image encoding result; and inputting the infrared image encoding result into the spiking neural network for training.

[0010] In a second aspect, some embodiments of the present disclosure provide an infrared image encoding apparatus for a spiking neural network, comprising: a determination unit configured to determine a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt an input of the spiking neural network; an enhancement range screening unit configured to determine at least one enhancement range to be enhanced in the gray scale range; a parameter generation unit configured to generate an enhancement parameter corresponding to each of the at least one enhancement range; a function construction unit configured to construct a hybrid mapping function according to the gray scale range, the mapping range and the obtained set of enhancement parameters; an image mapping unit configured to input each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, thereby obtaining a mapped image; a pulse conversion unit configured to convert each mapped pixel value in the mapped image into a pulse sequence through a Bernoulli experiment, thereby obtaining a set of pulse sequences; a three-dimensional encoding unit configured to splice each pulse sequence in the set of pulse sequences to generate three-dimensional data, thereby obtaining a set of three-dimensional data as an infrared image encoding result; and a training input unit configured to input the infrared image encoding result into the spiking neural network for training.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a storage having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method as described in any implementation manner of the first aspect.

[0013] The above embodiments of this disclosure have the following beneficial effects: Infrared images obtained through the infrared image encoding methods of some embodiments of this disclosure reduce the time of the pseudo-color conversion stage in computer recognition tasks, and improve computational speed and related performance. Specifically, the reason for the decrease in computer computational speed and related performance is that real-time pseudo-color conversion and signal processing during computer deployment consume a large amount of computing resources, resulting in redundant computing power and wasted energy. Based on this, the infrared image encoding method for spiking neural networks of some embodiments of this disclosure first determines the grayscale range and mapping range of the original infrared image based on the image histogram corresponding to the original infrared image, wherein the mapping range is used to adapt to the input range of the spiking neural network. This avoids the problem of decreased training efficiency caused by the mismatch between the input data range and the spiking neural network, ensuring direct compatibility of the data format. Then, at least one enhancement range to be enhanced in the grayscale range is determined. This focuses on key regions, provides a target range for feature enhancement, and improves sensitivity to specific targets. Secondly, for each of the at least one enhancement range, enhancement parameters corresponding to the enhancement range are generated. This generates enhancement factors for each enhancement region, providing data basis for constructing the enhancement function. Next, a hybrid mapping function is constructed based on the aforementioned grayscale range, mapping range, and enhancement parameter set. This achieves a balance between global grayscale adaptation and local feature enhancement through linear and nonlinear transformations, dynamically optimizing the image range. Then, each pixel value from the original infrared image is input into the hybrid mapping function to generate mapped pixel values, resulting in a mapped image. This produces a contrast-enhanced image, separating key target regions from the original background and reducing noise interference during model training. Next, through a Bernoulli experiment, each mapped pixel value in the mapped image is converted into a pulse sequence, resulting in a pulse sequence set. This transforms continuous pixel values ​​into discrete pulse sequences, simulating the activation characteristics of biological neurons and reducing computational redundancy. Then, each pulse sequence in the pulse sequence set is concatenated to generate three-dimensional data, resulting in a three-dimensional dataset as the infrared image encoding result. This forms a spatiotemporal fusion encoding structure, preserving the features of the original image and adapting to the temporal processing requirements of the spiking neural network. Finally, the infrared image encoding result is input into the spiking neural network for training. Therefore, the spatiotemporal training characteristics of spiking neural networks can be directly utilized for training, eliminating the need for traditional pseudo-color conversion. In summary, by converting the original image pixels into a spatiotemporal coding structure to adapt to the training of spiking neural networks, the need for manual pseudo-color conversion can be reduced, thereby reducing computational redundancy and energy waste. Attached Figure Description

[0014] The above and other features, aspects and advantages of the present disclosure will become more apparent after a reading of the following detailed description together with the accompanying drawings. Like reference numerals refer to like elements throughout the drawings. It should be understood that the drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the disclosure.

[0015] Figure 1 is a flowchart of some embodiments of an infrared image encoding method for a spiking neural network according to the present disclosure;

[0016] Figure 2 is a structural schematic diagram of some embodiments of an infrared image encoding apparatus for a spiking neural network according to the present disclosure;

[0017] Figure 3 is a structural schematic diagram of an electronic device suitable for use to implement some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thoroughly and completely understood. It should be understood that the drawings of the present disclosure and the embodiments thereof are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that only parts of the drawings related to the present application are shown for the sake of convenience. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0020] It should be noted that the terms "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the terms "one", "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0024] Reference Figure 1, shows a flow 100 of some embodiments of the infrared image encoding method for a spiking neural network according to the present disclosure. The infrared image encoding method for a spiking neural network comprises the following steps:

[0025] Step 101, determining a gray scale range and a mapping range of the original infrared image according to an image histogram corresponding to the original infrared image.

[0026] In some embodiments, the execution subject (e.g., a computing device) of the above-mentioned infrared image encoding method for a spiking neural network can be a computer. Wherein the above-mentioned image histogram can be a chart of the distribution of all pixel gray scale values, with the horizontal axis being the gray scale value and the vertical axis being the number of pixels corresponding to the gray scale value. Wherein the above-mentioned mapping range is the range used to adapt the input of the spiking neural network. The above-mentioned gray scale range can be an interval composed of the minimum and maximum values of all gray scale values. The above-mentioned mapping range is to adjust the original gray scale range to the target interval through mathematical transformation to adapt to the input requirements of the spiking neural network. The spiking neural network can be a third-generation artificial neural network, which simulates the mechanism of biological neurons transmitting information through pulses.

[0027] Step 102, determining at least one enhancement range to be enhanced in the gray scale range.

[0028] In some embodiments, the above-mentioned execution subject can determine at least one enhancement range to be enhanced in the above-mentioned gray scale range. Wherein the enhancement range can be a sub-interval divided in the original gray scale range. In practice, the key areas or features that need to be highlighted in the gray scale range can be selected. In forest fire monitoring, the enhancement range can be the high-temperature area of the flame, the high gray scale value interval. For example, the gray scale value is in the range of 1200 to 1600.

[0029] In the process of using the technical solutions to solve the above-mentioned technical problem one, it is often accompanied by the following problems, noise interference: for complex scenes with multiple targets, when performing pseudo-color encoding mapping, background noise is easy to be misjudged as valid signal, resulting in noise interference and affecting the accuracy of the model. Increased computational complexity: when optimizing the boundary of the enhancement range, the simplified range fusion may cause the loss of gray scale features of small temperature difference targets for high-resolution infrared images.

[0030] To solve these problems, the conventional solution is generally: using fixed step uniform segmentation or pre-defined threshold to divide the enhancement range, sacrificing accuracy for computational efficiency. The range fusion is performed by simplifying the weighted distance, thereby reducing the complexity. However, the inventors consider that fixed step segmentation and simplified range fusion may cause the loss of key feature regions. We decided to use the following solution.

[0031] In some optional implementations of some embodiments, the execution subject can determine at least one enhancement range to be enhanced in the above-mentioned gray scale range, which can include the following steps:

[0032] Firstly, the gray scale range is obtained according to the gray scale histogram of the original infrared image. The gray scale range can be the range of the minimum gray scale and the maximum gray scale of the original infrared image. For example, the minimum gray scale can be 0 and the maximum gray scale can be 16383, and the gray scale range can be [0, 16383].

[0033] Secondly, the pixels of the original infrared image are traversed to obtain the number of pixels corresponding to each gray scale value as a pixel frequency set. The pixel frequency set can be a set of the number of pixels corresponding to each gray scale value. In practice, firstly, each pixel value in the original image is counted, and then the number of occurrences of each gray scale is counted as the frequency of the pixel.

[0034] Thirdly, each pixel frequency in the pixel frequency set is divided by the total pixel value of the original infrared image to generate a pixel probability density and obtain a pixel probability density set. The pixel probability density can be the ratio of the frequency of each gray scale value to the total number of pixels of the original image. For example, the total number of pixels of the original image can be 1 million, and the frequency of the gray scale value 1000 can be 5000, and the pixel probability density corresponding to the gray scale value 1000 can be 0.005.

[0035] Fourthly, the gray scale value corresponding to the maximum pixel density in the pixel probability density set is determined as the first initial boundary value. The first initial boundary value can be the gray scale with the highest frequency in the image.

[0036] Fifthly, the gray scale range is uniformly divided according to the first initial boundary value to obtain the remaining boundary values as a boundary value set. Each boundary value in the boundary value set can be the result of expanding the first initial boundary value by a multiple. Firstly, the gray scale range is divided according to the division step, and the division step can be a preset step. For example, the preset step can be 10. Then, the boundary value set is obtained by sequentially adding the step length starting from the first initial boundary value.

[0037] Sixthly, for each boundary value in the boundary value set, the following operation steps are performed:

[0038] A first sub-step is to determine a weighted distance of the boundary value and the weight of each of the gray scale values, wherein the weight is the probability density of the pixel corresponding to the gray scale value. In practice, the weighted distance can be an index for measuring the difference between the boundary value and the surrounding gray scale values. First, the neighborhood radius and the neighborhood range of the boundary value are determined. Then, the gray scale values in the neighborhood are traversed, the distance from the boundary value is calculated and multiplied by the corresponding weight to obtain the weighted distance.

[0039] A second sub-step is to optimize the boundary value according to the minimum distance of the weighted distance to obtain an optimized boundary value. The minimum distance optimization can be to determine the optimal solution by finding the minimum value of the objective function, or to select the minimum position of the weighted distance as the optimization result by traversing the neighborhood range of the boundary value.

[0040] A seventh step is to sort the optimized boundary value set and merge the boundaries with a neighborhood distance less than a preset threshold to obtain an enhanced sub-range boundary set. The sorting can be ascending sorting. First, the boundary value set is sorted in ascending order to obtain a sorted queue. Then, the adjacent boundary values in the sorted queue are traversed to obtain a distance, and if the distance is less than the preset threshold, the boundary values are merged and replaced by the average value. For example, the sorted boundary values can be {0, 500, 750, 1200, 16383}, and the preset threshold can be 300, so the distance between 500 and 750 is 250, which is less than the preset threshold 300, and the boundary values are merged and replaced by the average value 625.

[0041] An eighth step is to perform the following operation steps for each enhanced sub-range boundary in the enhanced sub-range boundary set:

[0042] A third sub-step is to correct the enhanced sub-range boundary to the lower limit value of the gray scale range in response to the enhanced sub-range boundary being less than the lower limit value of the gray scale range. For example, the enhanced sub-range boundary can be {-50, 800, 16383}, and then -50 is less than 0, so -50 is replaced by 0.

[0043] A fourth sub-step is to correct the enhanced sub-range boundary to the upper limit value of the gray scale range in response to the enhanced sub-range boundary being greater than the upper limit value of the gray scale range. For example, the enhanced sub-range boundary can be {50, 800, 17000}, and then 17000 is greater than 16383, so 17000 is replaced by 16383.

[0044] In the ninth step, based on the above gray scale range, the enhancer range boundary is taken as the boundary of the enhancer range, and the enhancement range is obtained. The enhancement range can be a range of image processing intervals according to task requirements. In practice, the task requirement can be feature enhancement. First, the adjacent boundary values are traversed to generate a closed interval as a single interval range. Then it is verified whether the interval range is within the original gray scale range, and determined as the enhancement range.

[0045] The first step to the ninth step solve the technical problem mentioned in the background art, that is, when manually annotating the infrared image dataset, the invisible infrared signal intensity is mapped through the RGB / HSV visible light channel for pseudo-color encoding, which may mask the original gray scale features and exacerbate noise interference in model training, resulting in a decrease in accuracy. The factors leading to the above technical problem are often as follows: in a multi-target complex scene, due to inaccurate division of gray scale sub-ranges (for example, the high-temperature region and the background boundary are blurred), the pseudo-color encoding mapping is easy to misjudge the noise as a feature signal. For high-resolution infrared images, domain merging based on fixed threshold may result in loss of key feature region information. For example, a small temperature difference target. Therefore, the present disclosure designs a dynamic boundary optimization strategy, introduces a pixel probability density weighted domain difference calculation, and combines unsupervised clustering to pre-divide the initial boundary value, to improve the division accuracy of the gray scale sub-range. An adaptive merging threshold is introduced to merge and correct the boundary value beyond the boundary, to ensure the integrity of the features and retain the small temperature difference target.

[0046] In step 103, for each of the at least one enhancement range, an enhancement parameter corresponding to the enhancement range is generated.

[0047] In some embodiments, the execution subject can generate an enhancement parameter corresponding to each of the at least one enhancement range. The enhancement parameter can be a mathematical coefficient for adjusting the pixel value in the enhancement range.

[0048] As an example, the enhancement coefficient can be a contrast stretching coefficient, which maps the selected enhancement range to the enhancement range range, and the enhancement range [800, 1200] is mapped to the enhancement range [0, 255].

[0049] In some optional implementations of some embodiments, the execution subject can generate an enhancement parameter corresponding to the enhancement range, which can include the following steps:

[0050] In the first step, the lower limit value and the upper limit value corresponding to the enhancement range are obtained. For example, the enhancement range can be [0, 750], and the corresponding lower limit value can be 0 and the upper limit value can be 750.

[0051] Second, based on the lower limit and the upper limit, a mean value and a variance are obtained. The mean value can be the center position of the data distribution. The variance can be a description of the degree of dispersion of the data distribution. In infrared image coding, the mean value can be the middle value of the enhancement range. The variance can be a dynamic indicator for controlling the enhancement range and the enhancement intensity. For example, the variance can be one sixth of the difference between the lower limit of the upper limit of the enhancement range.

[0052] Third, the mean value and the variance are determined as the enhancement parameters. For example, the enhancement range can be [750, 5000], the mean value can be 2875, and the variance can be 3.0×10 6 The enhancement parameters can be 2875 and 3.0×10 6 .

[0053] Step 104, constructing a hybrid mapping function according to the gray scale range, the mapping range, and the obtained enhancement parameter set.

[0054] In some embodiments, the execution subject can construct a hybrid mapping function according to the gray scale range, the mapping range, and the obtained enhancement parameter set. The hybrid mapping function can be a composite function that combines global gray scale mapping and local enhancement functions.

[0055] As an example, the hybrid mapping function can be the superposition of the global mapping function and the local enhancement function.

[0056] In forest fire monitoring, the hybrid mapping function can be a piecewise function that selects the enhancement range and the original range of the infrared image for different temperature regions. For example, the original gray scale range can be [0, 16383], the target compression range can be [0, 255], the high temperature region can be [1200, 1600], the low temperature region can be [0, 800], the global mapping function can be a linear function that maps the gray scale range to the target compression range, and the local enhancement function can be a function that enhances the high temperature region and the low temperature region.

[0057] In some optional implementations of some embodiments, the hybrid mapping function can be a maximum function of the output value of a linear mapping function and the output value of a Gaussian enhancement function, where the linear mapping function is a linear function that linearly maps the gray scale range to the mapping range. For example, the linear function can be in the following form:

[0058]

[0059] O L may be the lower limit of the original image gray scale range, O U may be the upper limit of the original image gray scale range, P LP U P n x can be a pixel value in the image, B(x) can be a linear mapping value of a gray value in the original image after linear mapping.

[0060] The value range of the linear function is limited to the mapping range. The slope of the linear function is the ratio of the first difference value and the second difference value, the first difference value is the difference between the upper limit value of the mapping range and the lower limit value of the mapping range, and the second difference value is the difference between the upper limit value of the gray range and the lower limit value of the gray range. For example, the slope can be in the following form:

[0061]

[0062] The intercept of the linear function is the lower limit value of the mapping range. The Gaussian enhancement function is a nonlinear Gaussian function for gray value enhancement in the at least one enhancement range. For example, the nonlinear Gaussian function can be in the following form:

[0063]

[0064] μ n σ n σ n f

[0065] The value range of the Gaussian function is the mapping range. The offset of the Gaussian function is the lower limit value of the mapping range. The scaling factor of the Gaussian function is the difference between the upper limit value of the mapping range and the lower limit value of the mapping range. For example, the scaling factor can be P L -P U .

[0066] As an example, the hybrid mapping function can be in the following form:

[0067] f(x)=max{B(x),f1(x),f2(x),...,f M (x),x∈D}

[0068] max{} can be the maximum value of all functions in the parentheses. M is the number of divided enhancement groups, f(x) can be for each gray value x, calculate the linear mapping value B(x) and all enhancement functions f n (x), take the maximum value as the final mapping result.

[0069] Step 105, input each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, and obtain a mapped image.

[0070] In some embodiments, the execution subject can input each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, and obtain a mapped image. The mapped pixel value can be a pixel value of the original infrared image, which is obtained by the hybrid mapping function. The mapped image can be an image output after processing all pixel values of the original image by the hybrid function.

[0071] In forest fire monitoring, the original infrared image can be a 14-bit grayscale image collected by an infrared camera, the pixel value range of the high-temperature flame area can be within [1200, 1600], the pixel value range of the temperature of the background trees or ground can be within [0, 800], and the pixel value of the super-high-temperature flame can be greater than 1600. The hybrid mapping function can map the above ranges to a target range, where the target range can be [0, 255], the pixel value range of the mapped high-temperature flame area can be [150, 255], the pixel value range of the temperature of the mapped background trees or ground can be [0, 150], and the pixel value of the mapped super-high-temperature flame can be 255. The mapped pixels are arranged in the spatial form of the original image to obtain a mapped image.

[0072] Step 106, convert each mapped pixel value in the mapped image into a pulse sequence by Bernoulli experiment to obtain a pulse sequence set.

[0073] In some embodiments, the execution subject can convert each mapped pixel value in the mapped image into a pulse sequence by Bernoulli experiment to obtain a pulse sequence set. The pulse sequence can be a binary sequence composed of a single pixel and multiple time steps. In practice, the time step can be the number of experiments performed by the Bernoulli experiment. As an example, the Bernoulli experiment can be 4 times, and the single pixel pulse sequence can be [0, 1, 1, 0].

[0074] In some optional implementations of some embodiments, the execution subject can convert each mapped pixel value in the mapped image into a pulse sequence by Bernoulli experiment, which can include the following steps:

[0075] First, determine the lower limit value and the upper limit value of the enhancement range corresponding to the mapped pixel value. The lower limit value can be the grayscale boundary value of the lower limit of the enhancement range. The upper limit value can be the grayscale value of the upper limit of the enhancement range. In practice, first, traverse all the enhancement range set to find the range that meets the current mapped pixel value. Then, determine the lower limit value and the upper limit value of the range.

[0076] The second step is to determine the single-step probability corresponding to the mapped pixel value based on the lower limit and upper limit values ​​mentioned above. The single-step probability is the ratio of the first difference to the second difference. The first difference is the difference between the pixel value and the lower limit of the enhancement range, and the second difference is the difference between the upper limit of the enhancement range and the lower limit of the enhancement range.

[0077] In practice, the single-step probability can be the probability of generating a pulse at each time step, calculated based on the relative position of the pixel value within the enhancement range.

[0078] As an example, the enhancement range can be [5000, 16383], and the mapped pixel value can be 10000. Then the first difference can be 10000-5000=5000, the second difference can be 16383-5000=11383, and the single-step probability can be 5000 / 11383≈0.44.

[0079] The third step involves determining the following steps for each of Bernoulli's experiments:

[0080] The first sub-step involves generating random numbers corresponding to the above experiment. These random numbers can be uniformly distributed random variables within the interval [0, 1) to simulate the randomness of Bernoulli's experiment. For example, the random number could be 0.40.

[0081] The second sub-step, in response to the random number being less than the single-step probability, determines the experimental result as positive. A positive result can be an event judged as successful in a Bernoulli experiment, corresponding to an impulse of 1. For example, the random number could be 0.40, and the single-step probability could be 0.44. Therefore, 0.40 < 0.44, and the impulse is 1.

[0082] The third sub-step involves determining the experimental result of the experiment to be negative in response to the random number being no less than the single-step probability. A negative result can be an event judged as a failure in a Bernoulli experiment, with a corresponding impulse of 0. For example, the random number could be 0.60, and the single-step probability could be 0.44. Therefore, 0.60 > 0.44, and the impulse is 0.

[0083] The fourth step is to summarize the experimental results corresponding to the Bernoulli experiments described above to obtain the pulse sequence. The pulse sequence can be a sequence composed of the Bernoulli experiment results of the image pixels at multiple time steps.

[0084] Step 107: Each pulse sequence in the pulse sequence set is spliced ​​together to generate three-dimensional data, resulting in a three-dimensional dataset, which serves as the infrared image encoding result.

[0085] In some embodiments, the execution subject can stitch each pulse sequence in the pulse sequence set to generate three-dimensional data, obtaining a three-dimensional data set as the infrared image coding result. The pulse sequence set can be a set of pulse sequences of all pixels in the mapping image, and the three-dimensional data can be a data structure formed by the width, height, and time step of the original infrared image.

[0086] In practice, each pulse sequence in the pulse sequence set corresponds to a pulse record of a pixel at multiple time steps, which can be in the form of a binary array. The data structure can be in the form of a three-dimensional cube formed by the width, height, and time step of the original infrared image.

[0087] In some optional implementations of some embodiments, the execution subject can stitch each pulse sequence in the pulse sequence set to generate three-dimensional data, obtaining a three-dimensional data set, which can include the following steps:

[0088] First, obtain the width and height of the pixels of the original infrared image, and the length of each pulse sequence. The width and height can be the horizontal and vertical resolution of the image, the number of columns and rows of the image. The length of the pulse sequence can be the number of Bernoulli experiments.

[0089] As an example, first, read the width and height of the original infrared image using an image processing library. Finally, determine the number of experiments according to the Bernoulli experiment design or requirements. For example, the resolution of the original image can be 3x2 (width w, height h). The pulse sequence length (n) can be 3.

[0090] Second, construct an initial three-dimensional matrix according to the width, height, and length. As an example, create a three-dimensional zero array with shape (width, height, length).

[0091] Third, for each pulse sequence in the pulse sequence set, perform the following operation steps:

[0092] First sub-step, determine the pixel position of the original infrared image corresponding to the pulse sequence, obtaining a fill position, wherein the fill position includes a first fill position in the width direction of the original infrared image and a second fill position in the height direction of the original infrared image. For example, the fill position can be the coordinates of the pixel corresponding to the pulse sequence in the image (first fill position, second fill position).

[0093] As an example, by row priority or column priority order, traverse each pixel in the image to obtain the fill position.

[0094] A second sub-step, according to the filling position, sequentially filling the pulse sequence into the matrix position corresponding to the initial three-dimensional matrix in time order, wherein the matrix position is the time step corresponding to the depth direction, and corresponds to a pulse value in the pulse sequence.

[0095] As an example, the pulse sequence of pixel (i, j) can be [1, 0, 1]. The time step can be 3. Then the matrix position (i, j) corresponding to the three-dimensional matrix M can be, time step k = 0, M[i, j, 0] = 1. Time step k = 1, M[i, j, 1] = 0. Time step k = 2, M[i, j, 2] = 1.

[0096] A fourth step, by filling all pulse sequences, determining the initial three-dimensional matrix as a three-dimensional data filled with pulse sequences of multiple time steps.

[0097] As an example, first, traverse all pixels to ensure that the pulse sequence corresponding to each pixel has been filled. Finally, check whether there is a vacancy in the three-dimensional matrix for filling. For example, the filled three-dimensional matrix can be M = [[[1, 0, 1], [0, 1, 0]], [[1, 1, 0], [0, 0, 1]], [[0, 1, 1], [1, 0, 0]]].

[0098] A fifth step, determining the three-dimensional data as the three-dimensional data set.

[0099] Step 108, inputting the infrared image encoding result into the spiking neural network for training.

[0100] In some embodiments, the execution subject can input the infrared image encoding result into the spiking neural network for training. As an example, first, determine that the infrared image encoding result is a w x h x n cubic structure, the w direction and the h direction are the width direction and the height direction of the original infrared image, and the n direction is the time direction of the pulse generation. Secondly, taking the time direction as the input direction of the neural network, generating corresponding n w x h size pulse maps. Finally, input the pulse map into the pulse network for training.

[0101] In the process of adopting the technical solutions to solve the above technical problem two, the real-time pseudo-color conversion and signal processing in the computer deployment stage often consume a large amount of computing power. The training of the traditional spiking neural network requires strict synchronization of time steps for the input of three-dimensional space-time data, resulting in low utilization of hardware resources, long training time, and inability to take advantage of parallel computing.

[0102] To solve these problems, the conventional solution is generally to downsample high-resolution infrared images to reduce resolution and data volume to reduce load. Pseudo-color encoding conversion and model training are performed in stages to avoid real-time conversion overhead. However, the inventors consider that downsampling may cause loss of gray-scale information of small temperature difference targets and decrease of model generalization ability. Pseudo-color encoding conversion and model training in stages cannot meet the real-time inference requirements of dynamic scenes, such as unmanned aerial vehicle inspection. We decided to adopt the following solution.

[0103] In some optional implementations of some embodiments, the above execution subject can input the above infrared image encoding result into a spiking neural network for training, which can include the following steps:

[0104] First, configure the above infrared image encoding result as a spatio-temporal cube, wherein the spatio-temporal cube includes the width, height and time step of the original infrared image, and the time step is the number of Bernoulli experiments. The spatio-temporal cube can be a three-dimensional data cube with a shape of width (w) x height (h) x time step (n).

[0105] Second, according to the spatio-temporal cube, input the two-dimensional pulse graph of each time step in time sequence as the time sequence input of the spiking neural network, wherein the input data of each time step is a two-dimensional matrix composed of pulse values of all pixels at the current time step. The two-dimensional pulse graph can be a width (w) x height (h) matrix corresponding to each time step (k), and the elements in the matrix are the pulse values of all pixels at the current time step. The time sequence input is the input processing of the spiking neural network according to the time step, which simulates the dynamic response of the biological neural network.

[0106] Third, in the training process, an event-driven spiking neuron model is used to dynamically adjust the membrane potential and synaptic weight of the neuron. The event-driven spiking neuron model can be a neuron that only updates the membrane potential when receiving an input pulse, otherwise it remains silent. The membrane potential can be a new potential value calculated based on the input pulse and the current potential. The synaptic weight can be the connection weight between neurons adjusted according to the pulse timing correlation. The pulse timing correlation can be the STDP (Spike Timing Dependent Plasticity) rule.

[0107] In the fourth step, the spatio-temporal cube data is divided into multiple sub-blocks and distributed to a neuromorphic computing chip for distributed processing to accelerate the training process of the spiking neural network, in combination with an asynchronous parallel computing framework. The asynchronous parallel computing framework can divide the data into independent sub-blocks, and each sub-block is processed in parallel on different computing units without time synchronization. The neuromorphic computing chip can be designed for spiking computation and support event-driven and low power consumption, such as the Loihi chip or the TrueNorth chip.

[0108] As an example, first, the data is divided into multiple sub-blocks. Second, each sub-block is assigned to a core of the chip for independent processing of spiking events. Finally, the output pulses or weight update results are aggregated.

[0109] The first to fourth steps solve the second technical problem mentioned in the background that the real-time pseudo-color conversion and signal processing in the computer deployment stage occupy a large amount of computing resources, causing redundant computing power and energy waste. The factors that cause the above technical problem are often as follows: in a multi-target complex scene, due to inaccurate division of the gray scale sub-range (for example, the high temperature region and the background boundary are blurred), the pseudo-color coding mapping is easy to misjudge the noise as a feature signal. For high-resolution infrared images, based on fixed threshold domain merging, the key feature region information may be lost. For example, a small temperature difference target.

[0110] Therefore, the present disclosure designs a spatio-temporal three-dimensional structure and a one-step parallel computing architecture, which divides the high-dimensional spiking sequence into independent sub-blocks by constructing a three-dimensional spatio-temporal data structure (width x height x time step), and realizes asynchronous distributed parallel processing by using the event-driven characteristics of the neuromorphic computing chip (for example, the Loihi chip), breaking through the limitation of traditional time synchronization, and significantly improving the computing efficiency and resource utilization.

[0111] Optionally, the execution subject can further perform the following steps:

[0112] In the first step, the training result of the spiking neural network is obtained to obtain an infrared image spiking model. The infrared image spiking model can be a model obtained by training the pulse sequence corresponding to the infrared image based on the spiking neural network.

[0113] As an example, first, the trained model file (for example, the model weight file) can be loaded from a storage device (for example, a hard disk). Second, the model file is loaded, and the weight parameters obtained by training are loaded into the model network structure to construct a usable infrared image spiking model.

[0114] Second, input the original infrared image into the infrared image pulse model to obtain a pulse result sequence. The pulse result sequence can be a three-dimensional tensor (width x height x time step) of pulse firing conditions at each time step output by the pulse neural network.

[0115] Third, decouple the dimensions of the pulse result sequence and determine the corresponding pulse frequency in each dimension to obtain a dimensional pulse quantity. The dimensional decoupling can be to decompose the pulse result sequence into two dimensions of height and width in space, and keep the time step as an independent dimension in time. The pulse frequency can be the number of pulse firings in a unit of time or a unit of space in a specific dimension. The dimensional pulse quantity can be the pulse frequency information obtained by statistics in different dimensions.

[0116] As an example, first, the pulse result sequence is decoupled in space and time to obtain spatial and temporal dimensions. Second, in each spatial dimension, the total number of pulses in the entire time length is counted at the spatial position. The pulse frequency in the spatial dimension is obtained by dividing the total number of pulses by the time step. In each time step, the total number of pulses at the corresponding spatial position is counted. The average pulse frequency of each time step is obtained by dividing the total number of pulses by the total number of spatial pixels. Finally, the pulse frequency in the spatial dimension and the average pulse frequency in the temporal dimension are taken as the dimensional pulse quantity.

[0117] Fourth, according to the three-dimensional data set, the dimensional pulse quantity is mapped to the relative position to obtain a relative position offset. The relative position mapping can be to map the pulse frequency information to the position change in the actual three-dimensional space. The relative position offset can be the offset of the target relative to the current camera center position.

[0118] As an example, first, load the mapping relationship of image pixel coordinates to three-dimensional space coordinates from the three-dimensional data set. Filter the positions with high pulse frequency in the dimensional pulse quantity as target pixel coordinates. Second, convert the angle (e.g., azimuth) in the actual space according to the target pixel coordinates and the mapping relationship. Convert the coordinates corresponding to the current camera to the angle in the actual space. Calculate the difference between the two angles to obtain the relative position offset.

[0119] Fifth, generate an offset instruction from the relative position offset to obtain a camera control instruction. The offset instruction can be an instruction to control the rotation direction (e.g., horizontal and vertical) and the rotation amount (e.g., angle and step) of the camera. The camera control instruction can be a string instruction to specifically control the rotation of the camera.

[0120] As an example, the direction of the camera rotation can be determined according to the relative position offset. And the direction of the rotation is translated into the required instruction format for the camera control.

[0121] In the sixth step, the camera control instruction is sent to the camera control terminal to mobilize the infrared camera. The camera control terminal can be a hardware device (for example, a cloud controller) that receives the control instruction and drives the infrared camera to rotate.

[0122] As an example, the camera control instruction can be sent through the communication connection (for example, serial port and network port) of the camera control terminal. The camera control terminal can transmit the control instruction by calling the interface of the infrared camera to drive the infrared camera.

[0123] The above various embodiments of the present disclosure have the following beneficial effects: the infrared image obtained by the infrared image encoding method of some embodiments of the present disclosure has reduced time for pseudo-color conversion in computer recognition tasks, and the calculation speed and related performance are improved. Specifically, the reason for the decline in the calculation speed and related performance of the computer is that real-time pseudo-color conversion and signal processing in the computer deployment stage occupy a large amount of computing resources, causing redundant computing power and energy waste. Based on this, the infrared image encoding method for the pulse neural network of some embodiments of the present disclosure first determines the gray scale range and the mapping range of the original infrared image according to the image histogram corresponding to the original infrared image, wherein the mapping range is used to adapt the range of the input of the pulse neural network. In this way, the problem of training efficiency decline caused by the mismatch between the input data range and the pulse neural network is avoided, and the direct compatibility of the data format is ensured. Then, at least one enhancement range to be enhanced in the gray scale range is determined. In this way, the key area is focused on to provide a target range for feature enhancement and improve the sensitivity to specific targets. Secondly, for each enhancement range in the at least one enhancement range, an enhancement parameter corresponding to the enhancement range is generated. In this way, an enhancement factor is generated for each enhancement area to provide data basis for constructing an enhancement function. Then, a hybrid mapping function is constructed according to the gray scale range, the mapping range and the obtained set of enhancement parameters. In this way, through linear and nonlinear changes, the balance between global gray scale adaptation and local feature enhancement is achieved, and the range of the image is dynamically optimized. After that, each pixel value in the original infrared image is input into the hybrid mapping function to generate a mapping pixel value and obtain a mapping image. In this way, the image after contrast enhancement is obtained, so that the key target area is separated from the original background, and the noise interference during model training is reduced. Then, through the Bernoulli experiment, each mapping pixel value in the mapping image is converted into a pulse sequence to obtain a pulse sequence set. In this way, the continuous pixel value is converted into a discrete pulse sequence to simulate the activation characteristics of biological neurons and reduce the redundancy during calculation. Then, each pulse sequence in the pulse sequence set is spliced to generate a three-dimensional data and obtain a three-dimensional data set as an infrared image encoding result. In this way, a time-space fusion encoding structure is formed, the features of the original image are preserved, and the demand of the pulse neural network for time sequence processing is met. Finally, the infrared image encoding result is input into the pulse neural network for training. In this way, the time-space training characteristics of the pulse neural network can be directly used for training, and the traditional pseudo-color conversion link is omitted. In summary, by converting the original image pixels into an encoding structure with time-space sequence to adapt the training of the pulse neural network, the pseudo-color conversion of artificial labeling can be reduced, thereby reducing the redundancy of computing power and energy waste.

[0124] Further reference Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an infrared image coding apparatus for a spiking neural network, which apparatus embodiments correspond to those method embodiments shown in Figure 1 The infrared image coding apparatus for a spiking neural network can be specifically applied in various electronic devices.

[0125] As shown in Figure 2 An infrared image coding apparatus 200 for a spiking neural network comprises a determination unit 201, an enhanced range screening unit 202, a parameter generation unit 203, a function construction unit 204, an image mapping unit 205, a spiking conversion unit 206, a three-dimensional coding unit 207 and a training input unit 208. The determination unit 201 is configured to determine a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt the input of the spiking neural network. The enhanced range screening unit 202 is configured to determine at least one enhanced range to be enhanced in the gray scale range. The parameter generation unit 203 is configured to generate an enhanced parameter corresponding to each enhanced range in the at least one enhanced range. The function construction unit 204 is configured to construct a hybrid mapping function according to the gray scale range, the mapping range and the obtained set of enhanced parameters. The image mapping unit 205 is configured to input each pixel value in the original infrared image into the hybrid mapping function to generate a mapping pixel value, thereby obtaining a mapping image. The spiking conversion unit 206 is configured to convert each mapping pixel value in the mapping image into a pulse sequence through Bernoulli experiment, thereby obtaining a set of pulse sequences. The three-dimensional coding unit 207 is configured to splice each pulse sequence in the set of pulse sequences to generate three-dimensional data, thereby obtaining a set of three-dimensional data as an infrared image coding result. The training input unit 208 is configured to input the infrared image coding result into the spiking neural network for training.

[0126] It can be understood that the units described in the infrared image coding apparatus 200 for a spiking neural network correspond to the respective steps in the methods described with reference to Figure 1 The operations, features and beneficial effects described above for the methods also apply to the infrared image coding apparatus 200 for a spiking neural network and the units contained therein, and will not be repeated here.

[0127] Reference is made below to Figure 3 which shows a structural schematic diagram of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0128] As shown in FIG. 3, Figure 3 The electronic device 300 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or loaded into a random access memory (RAM) 303 from a storage device 308. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0129] In general, the following devices can be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other devices to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it is understood that not all of the shown devices are required to be implemented or present. More or fewer devices can alternatively be implemented or present. Figure 3 Each block shown in FIG. 3 can represent a device or, as appropriate, a plurality of devices.

[0130] In particular, processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present disclosure. For example, some embodiments of the present disclosure include a computer program product including a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network through the communication devices 309, or installed from the storage devices 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-described functions defined in the methods of some embodiments of the present disclosure are performed.

[0131] Note that the computer readable medium in some embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus or device, or that can be used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present disclosure, the computer readable signal medium can include a computer readable program code propagated on or through a carrier wave, in baseband or as part of a carrier wave. Such propagated signals can take a variety of forms, including but not limited to electro-magnetic signals, optical signals or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to wire, cable, RF (radio frequency), infrared or any suitable combination thereof.

[0132] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0133] The computer readable medium can be included in the electronic device; or can exist independently of the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: determine a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt an input of a spiking neural network; determine at least one enhancement range to be enhanced in the gray scale range; generate an enhancement parameter corresponding to each of the at least one enhancement range; construct a hybrid mapping function according to the gray scale range, the mapping range, and the obtained set of enhancement parameters; input each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, and obtain a mapped image; convert each mapped pixel value in the mapped image into a pulse sequence through a Bernoulli experiment, and obtain a set of pulse sequences; splice each pulse sequence in the set of pulse sequences to generate three-dimensional data, and obtain a set of three-dimensional data as an infrared image encoding result; and input the infrared image encoding result into the spiking neural network for training.

[0134] Computer program code for carrying out operations of some embodiments of the disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0135] The computer program product of the first aspect can include a computer readable storage medium. The computer readable storage medium can include instructions. The instructions can include one or both of: instructions for causing a computer to implement a method as described above; and instructions for causing a computer to operate based on a system as described above. The computer readable storage medium can include one or more of: a magnetic disk; a magnetic tape; a CD-ROM (compact disk read only memory); a semiconductor memory; and an optical disk.

[0136] The units described in some embodiments of the present disclosure can be implemented by means of software, or by means of hardware. The described units can also be provided in a processor, for example, a processor can be described as including a determination unit, an enhanced range screening unit, a parameter generation unit, a function construction unit, an image mapping unit, a pulse conversion unit, a three-dimensional encoding unit, and a training input unit. In some cases, the names of these units do not constitute a limitation on the units themselves, for example, the determination unit can also be described as "a unit for determining the gray scale range and the mapping range of the original infrared image according to the image histogram corresponding to the original infrared image".

[0137] The functions described above in the detailed description can be performed by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0138] The above description is merely exemplary of some preferred embodiments of the present disclosure and of the application of the principles thereof, and the scope of protection that protects the present disclosure is not limited to the specific embodiments described herein, but rather extends to any alternative embodiments thereof based on the principles described herein and the scope of equivalents to the technical features described herein. For example, the technical features described above can be replaced with other technical features disclosed in the embodiments of the present disclosure (but not limited to) having similar functions to form other technical solutions.

Claims

1. An infrared image encoding method for a spiking neural network, comprising: determining a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt an input of a spiking neural network; determining at least one enhancement range in the gray scale range to be feature enhanced; generating an enhancement parameter corresponding to each of the at least one enhancement range; constructing a hybrid mapping function according to the gray scale range, the mapping range and the obtained set of enhancement parameters; inputting each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value, thereby obtaining a mapped image; converting each mapped pixel value in the mapped image into a pulse sequence through a Bernoulli experiment, thereby obtaining a set of pulse sequences; splicing each pulse sequence in the set of pulse sequences to generate three-dimensional data, thereby obtaining a set of three-dimensional data as an infrared image encoding result; inputting the infrared image encoding result into a spiking neural network for training.

2. The method of claim 1, wherein, The method further comprises: obtaining a training result of the spiking neural network, thereby obtaining an infrared image spiking model; inputting the original infrared image into the infrared image spiking model, thereby obtaining a pulse result sequence; dimensionally decoupling the pulse result sequence and determining a corresponding pulse frequency on each dimension, thereby obtaining a dimensional pulse quantity; mapping the dimensional pulse quantity according to the set of three-dimensional data, thereby obtaining a relative position offset; generating an offset instruction from the relative position offset, thereby obtaining a camera control instruction; sending the camera control instruction to a camera control terminal to mobilize an infrared camera.

3. The method of claim 1, wherein, The method further comprises: obtaining a lower limit value and an upper limit value corresponding to the enhancement range; obtaining a mean value and a variance based on the lower limit value and the upper limit value; determining the mean value and the variance as enhancement parameters.

4. The method of claim 1, wherein, The hybrid mapping function is a maximum value function of a linear mapping function output value and a Gaussian enhancement function output value, wherein the linear mapping function is a linear function that linearly maps the gray scale range to the mapping range; the linear function has a value range of the mapping range; a slope of the linear function is a ratio of a first difference value and a second difference value, the first difference value is a difference between an upper limit value of the mapping range and a lower limit value of the mapping range, and the second difference value is a difference between an upper limit value of the gray scale range and a lower limit value of the gray scale range; an intercept of the linear function is the lower limit value of the mapping range; the Gaussian enhancement function is a nonlinear Gaussian function that enhances a gray scale value in the at least one enhancement range; the nonlinear Gaussian function has a value range of the mapping range; an offset of the nonlinear Gaussian function is the lower limit value of the mapping range; and a scaling factor of the nonlinear Gaussian function is a difference between the upper limit value of the mapping range and the lower limit value of the mapping range.

5. The method of claim 1, wherein, The method further comprises: obtaining a lower limit value and an upper limit value corresponding to the enhancement range; obtaining a mean value and a variance based on the lower limit value and the upper limit value; determining the mean value and the variance as enhancement parameters. The hybrid mapping function is a maximum value function of a linear mapping function output value and a Gaussian enhancement function output value, wherein the linear mapping function is a linear function that linearly maps the gray scale range to the mapping range; the linear function has a value range of the mapping range; a slope of the linear function is a ratio of a first difference value and a second difference value, the first difference value is a difference between an upper limit value of the mapping range and a lower limit value of the mapping range, and the second difference value is a difference between an upper limit value of the gray scale range and a lower limit value of the gray scale range; an intercept of the linear function is the lower limit value of the mapping range; the Gaussian enhancement function is a nonlinear Gaussian function that enhances a gray scale value in the at least one enhancement range; the nonlinear Gaussian function has a value range of the mapping range; an offset of the nonlinear Gaussian function is the lower limit value of the mapping range; and a scaling factor of the nonlinear Gaussian function is a difference between the upper limit value of the mapping range and the lower limit value of the mapping range. The method further comprises: obtaining a lower limit value and an upper limit value corresponding to the enhancement range; obtaining a mean value and a variance based on the lower limit value and the upper limit value; determining the mean value and the variance as enhancement parameters. determining a lower limit value and an upper limit value of an enhancement range corresponding to the mapped pixel value; determining a single-step probability corresponding to the mapped pixel value based on the lower limit value and the upper limit value, wherein the single-step probability is a ratio of a first difference value and a second difference value, the first difference value being a difference between the pixel value and the lower limit value of the enhancement range, and the second difference value being a difference between the upper limit value of the enhancement range and the lower limit value of the enhancement range; for each experiment in the Bernoulli experiment, performing the following determination steps: generating a random number corresponding to the experiment; in response to the random number being less than the single-step probability, determining that an experimental result of the experiment is a positive result; in response to the random number not being less than the single-step probability, determining that the experimental result of the experiment is a negative result; summarizing each experimental result corresponding to the Bernoulli experiment to obtain the pulse sequence.

6. The method of claim 1, wherein, the splicing of each pulse sequence in the pulse sequence set to generate three-dimensional data to obtain a three-dimensional data set, comprising: obtaining a width and a height of a pixel of the original infrared image, and a length of each pulse sequence; constructing an initial three-dimensional matrix according to the width and the height, and the length; for each pulse sequence in the pulse sequence set, performing the following operation steps: determining a pixel position of the original infrared image corresponding to the pulse sequence to obtain a filling position, wherein the filling position includes a first filling position in the width direction of the original infrared image and a second filling position in the height direction of the original infrared image; according to the filling position, sequentially filling the pulse sequence into a matrix position corresponding to the initial three-dimensional matrix in time sequence, wherein the matrix position is a time step corresponding to the depth direction, corresponding to a pulse value in the pulse sequence; by filling all pulse sequences, determining the initial three-dimensional matrix as three-dimensional data filled by the pulse sequences of the multiple time steps; determining the three-dimensional data as the three-dimensional data set.

7. An infrared image coding device for a pulse neural network, comprising: a determination unit configured to determine a gray scale range and a mapping range of an original infrared image according to an image histogram corresponding to the original infrared image, wherein the mapping range is a range used to adapt an input of a pulse neural network; an enhancement range screening unit configured to determine at least one enhancement range to be enhanced in feature in the gray scale range; a parameter generation unit configured to generate an enhancement parameter corresponding to each enhancement range in the at least one enhancement range; a function construction unit configured to construct a hybrid mapping function according to the gray scale range, the mapping range and the obtained enhancement parameter set; an image mapping unit configured to input each pixel value in the original infrared image into the hybrid mapping function to generate a mapped pixel value to obtain a mapped image; a pulse conversion unit configured to convert each mapped pixel value in the mapped image into a pulse sequence through a Bernoulli experiment to obtain a pulse sequence set. a three-dimensional encoding unit configured to stitch each of the pulse sequences in the set of pulse sequences to generate three-dimensional data, resulting in a three-dimensional data set, as an infrared image encoding result; a training input unit configured to input the infrared image encoding result into a spiking neural network for training.

8. An electronic device, comprising: one or more processors; storage having stored thereon one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of claims 1-6.

9. A computer readable medium having stored thereon a computer program, wherein, the program that when executed by a processor implements the method of any of claims 1-6. the program that when executed by a processor implements the method of any of claims 1-6.

Images