Object recognition method and device, storage medium and electronic equipment

Abstract

The invention discloses an object recognition method and device, a storage medium and electronic equipment. The method comprises the steps that a to-be-recognized area is determined according to the pixel probability of each pixel point in a to-be-recognized image, and the pixel probability of the pixel points in the to-be-recognized area meets a preset probability judgment condition; visible light feature extraction operation and thermal sensing feature extraction operation are executed on the to-be-recognized area, visible light features and thermal sensing features of the to-be-recognized area are obtained, the visible light features comprise circularity features, texture features and high-frequency energy features, and the thermal sensing features comprise temperature distribution features and centroid features; performing pyroelectric feature extraction operation on the to-be-recognized image to obtain pyroelectric features of the to-be-recognized area; and determining an identification result according to the visible light feature, the thermal sensing feature and the pyroelectric feature. The technical problem that the accuracy of object recognition is low is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the computer field, and in particular, to an object recognition method and device, a storage medium, and an electronic device. BACKGROUND

[0002] In target detection under a complex light and background environment, the prior art faces the technical problem of low recognition accuracy. For example, in an image processing-based recognition method, a single feature or a sensor-based recognition system is often relied on. When facing a variable real scene, it is often difficult to accurately distinguish between a target and a non-target, especially when the target feature is not obvious or highly fused with the background, and the recognition performance cannot be guaranteed.

[0003] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0004] Embodiments of the present application provide an object recognition method and device, a storage medium, and an electronic device to at least solve the technical problem of low object recognition accuracy.

[0005] According to an aspect of an embodiment of the present application, an object recognition method is provided, including: determining a to-be-recognized region according to pixel probabilities of each pixel point in a to-be-recognized image, wherein the pixel probability of the pixel point is determined by a first chroma component and a second chroma component of the pixel point, and the pixel probabilities of the pixel points in the to-be-recognized region satisfy a preset probability discrimination condition; performing a visible light feature extraction operation and a thermal sensor feature extraction operation on the to-be-recognized region respectively to obtain a visible light feature and a thermal sensor feature of the to-be-recognized region, wherein the visible light feature includes a circularity feature, a texture feature, and a high-frequency energy feature, and the thermal sensor feature includes a temperature distribution feature and a centroid feature; and performing a pyroelectric feature extraction operation on the to-be-recognized image to obtain a pyroelectric feature of the to-be-recognized region; and determining a recognition result according to the visible light feature, the thermal sensor feature, and the pyroelectric feature, wherein the recognition result is used to indicate whether the to-be-recognized image contains a target object.

[0006] According to another aspect of the embodiments of the present application, an object recognition device is also provided, which comprises: a determination module configured to determine a to-be-recognized region according to pixel probabilities of each pixel point in a to-be-recognized image, wherein the pixel probability of each pixel point is determined according to a first chroma component and a second chroma component of the pixel point, and the pixel probabilities of the pixel points in the to-be-recognized region satisfy a preset probability discrimination condition; an execution module configured to perform a visible light feature extraction operation and a thermal sensing feature extraction operation on the to-be-recognized region respectively, to obtain a visible light feature and a thermal sensing feature of the to-be-recognized region, wherein the visible light feature comprises a circularity feature, a texture feature and a high-frequency energy feature, and the thermal sensing feature comprises a temperature distribution feature and a centroid feature; and perform a pyroelectric feature extraction operation on the to-be-recognized image, to obtain a pyroelectric feature of the to-be-recognized region; and an identification module configured to determine an identification result according to the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the identification result is used to indicate whether the to-be-recognized image contains a target object.

[0007] As an optional embodiment, the device is configured to determine the identification result according to the visible light feature, the thermal sensing feature and the pyroelectric feature by: fusing the visible light feature, the thermal sensing feature and the pyroelectric feature to obtain a target feature vector; and determining the identification result according to the target feature vector, wherein the identification result is used to indicate whether the to-be-recognized image contains a flame, and the target object comprises the flame.

[0008] As an optional embodiment, the device is configured to perform the visible light feature extraction operation and the thermal sensing feature extraction operation on the to-be-recognized region respectively, to obtain the visible light feature and the thermal sensing feature of the to-be-recognized region, by: obtaining a perimeter parameter and an area parameter of the to-be-recognized region; determining the circularity feature according to a product of the area parameter and the perimeter parameter; generating a gray level co-occurrence matrix according to the to-be-recognized region, and generating an energy parameter, an entropy parameter, a contrast parameter and a correlation parameter according to the gray level co-occurrence matrix; determining the texture feature based on the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter; and performing a frequency domain decomposition operation on the to-be-recognized region to obtain a plurality of subbands and a subband energy corresponding to each of the subbands, and determining the high-frequency energy feature based on the subband energy.

[0009] As an optional embodiment, the device is configured to determine the circularity feature according to the product parameter of the area parameter and the perimeter parameter by: determining a product of a preset parameter and the product parameter and a square of the perimeter parameter as a value of the circularity feature, wherein the preset parameter is a positive integer greater than 0.

[0010] As an optional embodiment, the device is configured to generate the gray level co-occurrence matrix according to the region to be identified by: obtaining a plurality of preset matrix direction parameters; and generating a plurality of target gray level co-occurrence matrices according to the plurality of matrix direction parameters and a preset distance parameter, wherein the gray level co-occurrence matrix comprises the plurality of target gray level co-occurrence matrices, and one of the matrix direction parameters corresponds to one of the target gray level co-occurrence matrices.

[0011] As an optional embodiment, the device is further configured to: generate the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter according to each of the target gray level co-occurrence matrices respectively, wherein one of the target gray level co-occurrence matrices corresponds to a group of target parameters comprising the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter; determine an energy mean value parameter and an energy standard deviation parameter according to the energy parameter in each of the group of target parameters; determine an entropy mean value parameter and an entropy standard deviation parameter according to the entropy parameter in each of the group of target parameters; determine a contrast mean value parameter and a contrast standard deviation parameter according to the contrast parameter in each of the group of target parameters; determine a correlation mean value parameter and a correlation standard deviation parameter according to the correlation parameter in each of the group of target parameters; and determine the value of the texture feature using the energy mean value parameter and the energy standard deviation parameter, the entropy mean value parameter and the entropy standard deviation parameter, the contrast mean value parameter and the contrast standard deviation parameter, and the correlation mean value parameter and the correlation standard deviation parameter.

[0012] As an optional embodiment, the device is configured to generate the gray level co-occurrence matrix according to the region to be identified by: obtaining a plurality of preset matrix direction parameters; and generating a plurality of target gray level co-occurrence matrices according to the plurality of matrix direction parameters and a preset distance parameter, wherein the gray level co-occurrence matrix comprises the plurality of target gray level co-occurrence matrices, and one of the matrix direction parameters corresponds to one of the target gray level co-occurrence matrices.

[0013] As an optional embodiment, the device is configured to perform a frequency domain decomposition operation on the to-be-identified region to obtain a plurality of subbands and a subband energy corresponding to each of the subbands, and determine the high-frequency energy feature based on the subband energy.

[0014] As an optional embodiment, the device is configured to perform a thermal sensing feature extraction operation on the to-be-identified region to obtain a thermal sensing feature by: acquiring a temperature of each of the pixel points in the to-be-identified region using a thermal imaging module; determining a temperature distribution feature according to a temperature mean parameter, a temperature variance parameter and a temperature scale parameter of the pixel points; determining a target pixel point set satisfying a preset temperature condition according to the temperature of each of the pixel points, and acquiring a mass point pixel point of the to-be-identified region, and determining the centroid feature according to the mass point pixel point and the target pixel point set.

[0015] As an optional embodiment, the device is configured to determine the centroid feature according to the mass point pixel point and the target pixel point set by: determining a value of the centroid feature as a positive integer 1 in a case where the mass point pixel point belongs to the target pixel point set; and determining the value of the centroid feature as 0 in a case where the mass point pixel point does not belong to the target pixel point set.

[0016] As an optional embodiment, the device is configured to perform a pyroelectric feature extraction operation on the to-be-identified image to obtain a pyroelectric feature of the to-be-identified region by: performing a pyroelectric feature extraction operation on the to-be-identified image using a first waveband filter, a second waveband filter and a third waveband filter to obtain a first pyroelectric feature, a second pyroelectric feature and a third pyroelectric feature, wherein the first waveband filter, the second waveband filter and the third waveband filter are all different; and determining the pyroelectric feature using the first pyroelectric feature, the second pyroelectric feature and the third pyroelectric feature.

[0017] According to still another aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is configured to perform the object recognition method when running.

[0018] According to a further aspect of the embodiments of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the object recognition method as above.

[0019] According to a further aspect of the embodiments of the present application, an electronic device is also provided, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the object recognition method as above by using the computer program.

[0020] In the embodiments of the present application, the to-be-recognized region is determined according to the pixel probability of each pixel point in the to-be-recognized image, wherein the pixel probability of the pixel point is determined by the first chroma component and the second chroma component of the pixel point, and the pixel probability of the pixel point in the to-be-recognized region satisfies a preset probability discrimination condition; the visible light feature extraction operation and the thermal sensing feature extraction operation are respectively performed on the to-be-recognized region to obtain the visible light feature and the thermal sensing feature of the to-be-recognized region, wherein the visible light feature comprises the circularity feature, the texture feature and the high-frequency energy feature, and the thermal sensing feature comprises the temperature distribution feature and the centroid feature; and the pyroelectric feature extraction operation is performed on the to-be-recognized image to obtain the pyroelectric feature of the to-be-recognized region; and the recognition result is determined according to the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the recognition result is used to represent whether the target object is contained in the to-be-recognized image, and through the multi-sensor information fusion and the deep feature learning, the object recognition accuracy is improved, so that the technical effect of stably and efficiently recognizing the target in a complex environment is achieved, and the technical problem of low accuracy in the object recognition by using the traditional single feature or sensor method is effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and the explanation of the present application, and do not constitute improper limitations on the present application. In the drawings:

[0022] Figure 1 is a schematic diagram of an application environment of an optional object recognition method according to the embodiments of the present application;

[0023] Figure 2 is a flowchart of an optional object recognition method according to the embodiments of the present application;

[0024] Figure 3 is a schematic diagram of an optional object recognition method according to the embodiments of the present application;

[0025] Figure 4is a schematic diagram of still another optional object recognition method according to an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of still another optional object recognition method according to an embodiment of the present application;

[0027] Figure 6 is a structural schematic diagram of an optional object recognition apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the protection scope of the present application.

[0029] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0030] The present application will be described below with reference to the embodiments:

[0031] According to an aspect of the embodiments of the present application, an object recognition method is provided. Optionally, in the present embodiment, the above-mentioned object recognition method can be applied in a hardware environment composed of a server 101 and a terminal device 103 as shown in Figure 1 Figure 1 ​As shown, the server 101 is connected with the terminal device 103 through a network, which can be used to provide services for the terminal device or the application program installed on the terminal device, and the application program 107 can be a video application program, an instant messaging application program, a browser application program, an education application program, a game application program, etc. The database 105 can be set on the server or independently of the server, which is used to provide data storage services for the server 101, for example, a game data storage server. The above network can include but is not limited to a wired network and a wireless network, wherein the wired network includes a local area network, a metropolitan area network and a wide area network, and the wireless network includes Bluetooth, WIFI and other wireless communication networks. The terminal device 103 can be a terminal configured with an application program, which can include but is not limited to at least one of the following: a mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a notebook computer, a tablet computer, a palm computer, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart television, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, an MR (MixedReality) terminal, etc. The above server can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0032] In combination Figure 1 As shown, the above object recognition method can be executed by an electronic device, which can be a terminal device or a server. The above object recognition method can be realized by the terminal device or the server respectively, or realized by the terminal device and the server together.

[0033] The above is only an example, and the present embodiment is not limited specifically.

[0034] Optionally, as an optional implementation manner, as shown in Figure 2 As shown, the above object recognition method includes:

[0035] S202, determining a to-be-recognized region according to pixel probabilities of each pixel point in a to-be-recognized image, wherein the pixel probability of the pixel point is determined by a first chroma component and a second chroma component of the pixel point, and the pixel probability of the pixel point in the to-be-recognized region satisfies a preset probability discrimination condition;

[0036] Optionally, in the embodiments of the present application, the initial image of the to-be-identified image can include but is not limited to a portrait, a pet, a mechanical device, a landscape, etc. The to-be-identified image can be obtained in the following ways, including but not limited to using an image acquisition device to collect images in real time and uploading them to an image storage device, or using an image acquisition device to collect a video, obtaining an initial image by intercepting video frames, uploading the initial image to an image storage device, or using an image acquisition device to collect a video, uploading the video to an image storage device, and obtaining an initial image by intercepting video frames from the image storage device. The image acquisition device can include but is not limited to a smartphone, a computer, a notebook, a smart wearable device, a video camera, etc. The image storage device can include but is not limited to a server, a computer, a notebook computer, a smartphone, a tablet computer, a digital camera, a scanner, a printer, etc.

[0037] Optionally, in the embodiments of the present application, the first chroma component and the second chroma component can be regarded as color information of the image in a specific color space. For example, in the YCbCr color model, the first chroma component is Cb and the second chroma component is Cr, which respectively represent the chroma components of blue and red. In addition, the probability discrimination condition refers to a set threshold or standard used to filter out target regions with high probability, which helps to narrow the scope of subsequent feature analysis and improve processing efficiency and recognition accuracy.

[0038] It should be noted that the first chroma component and the second chroma component are not limited to Cb and Cr. Depending on different application scenarios, they can also be the saturation and hue of the HSV color space, or the a* and b* components of the Lab color space, etc. As long as they can effectively represent the color difference in the image, they are applicable to the recognition framework mentioned in the present application to flexibly adapt to the recognition needs of different environments and targets.

[0039] Similarly, the probability discrimination condition can be determined according to the feature distribution of the target object in a specific chroma space, combined with experimental data or empirical values, which is not limited in the present application, aiming to achieve the best recognition performance.

[0040] For example, the to-be-identified image is first converted from the RGB color space to the YCbCr color space, where Y represents the luminance component of the channel, Cb represents the blue chroma component of the channel, and Cr represents the red chroma component of the channel. Assuming that the target object to be identified is a flame, the flame pixels in the (Cr, Cb) space basically conform to a two-dimensional normal distribution, so we can get:

[0041]

[0042] wherein, S Cr , S CbMean and standard deviation of Cr and Cb, respectively.

[0043] Then, the probability f(Cr, Cb) of each pixel point, i.e., the probability of the flame pixel, is obtained. When f(Cr, Cb) is greater than a threshold T, it means that the pixel point in the to-be-identified region needs to meet a preset probability discrimination condition, at which time the coordinates of the pixel point are recorded, and a plurality of recorded pixel points are determined as the flame region A(x, y). Wherein, T = 3e-15.

[0044] S204, visible light feature extraction operation and thermal sensing feature extraction operation are respectively performed on the to-be-identified region to obtain the visible light feature and the thermal sensing feature of the to-be-identified region, wherein the visible light feature includes the circularity feature, the texture feature and the high-frequency energy feature, and the thermal sensing feature includes the temperature distribution feature and the centroid feature; and pyroelectric feature extraction operation is performed on the to-be-identified image to obtain the pyroelectric feature of the to-be-identified region;

[0045] Optionally, in the embodiment of the present application, the circularity feature refers to the similarity between the shape of the target region and the ideal circle, the texture feature relates to the structure and pattern of the local region of the image, and the high-frequency energy feature reflects the energy distribution of the image in the high-frequency subband. The temperature distribution feature in the thermal sensing feature describes the temperature variation in the target region, and the centroid feature locates the center position of the target region. The pyroelectric feature is based on the response of the pyroelectric sensor to infrared radiation in a specific wavelength range, including but not limited to the radiation intensity information of three wavebands of 3.8 μm, 4.4 μm and 5.3 μm.

[0046] It should be noted that, for the visible light feature, in addition to the circularity, texture and high-frequency energy, other features such as color contrast and edge strength can also be included, and the specific selection depends on the characteristics of the target object and the application scenario; the temperature distribution and centroid feature of the thermal sensing feature can also be extended to temperature gradient and hot spot distribution, etc., to enhance the recognition effect. The waveband selection of the pyroelectric feature can be based on the radiation characteristics of the target in the infrared spectrum, which is not limited in the present application.

[0047] Exemplarily, the visible light feature extraction operation aims to analyze the characteristics of the to-be-identified region under the visible light spectrum, covering the circularity feature, the texture feature and the high-frequency energy feature. The thermal sensing feature extraction operation focuses on the temperature distribution and centroid feature of the target region, and captures thermal information using thermal imaging technology. The pyroelectric feature extraction operation uses the characteristics of the pyroelectric sensor to obtain radiation data of specific wavebands, further enriching the feature set.

[0048] S206, determining the recognition result according to the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the recognition result is used to represent whether the target object is contained in the to-be-identified image.

[0049] It should be noted that the combination of the above visible light features, thermal sensing features and pyroelectric features is not limited to simple weighted averaging, but can also include complex data fusion techniques such as machine learning models, neural networks or deep learning algorithms, which can learn the interdependence between features to improve recognition accuracy and robustness.

[0050] In addition, the generation mechanism of the recognition result can be diversified, which can be a simple threshold judgment or a probability-based decision framework. The specific implementation will be customized according to the type and magnitude of the features, the characteristics of the target object and the expected recognition accuracy, which is not limited in this application.

[0051] In an exemplary embodiment, taking the application scenario of a forest fire monitoring system as an example, the system first receives the image to be identified captured by the camera. Next, the system calculates the probability that each pixel belongs to the flame according to the first chroma component (Cr) and the second chroma component (Cb) of each pixel. By setting a probability discrimination threshold T, the system filters out all pixels with a probability greater than T, and defines the set of these pixels as the to-be-identified region.

[0052] After determining the to-be-identified region, the system performs visible light feature extraction and thermal sensing feature extraction operations respectively. For visible light feature extraction, the system calculates the circularity feature of the to-be-identified region to exclude non-flame circular light sources; extracts texture features to capture dynamic textures of the flame; and obtains high-frequency energy features through wavelet transform to identify high-frequency details of the flame. For thermal sensing feature extraction, the system uses thermal imaging sensors to obtain temperature distribution features to judge the temperature change in the region; at the same time, by calculating the centroid feature of the flame region, the center position of the flame is determined to further verify its possibility of being a flame.

[0053] The pyroelectric feature extraction operation is performed when the system further analyzes the infrared radiation characteristics. The pyroelectric sensor measures the radiation intensity at three specific wavebands (3.8 μm, 4.4 μm and 5.3 μm), thereby obtaining the pyroelectric features of the to-be-identified region. These features can enhance the system's ability to identify flames, especially in cases where visible light features are not obvious or environmental light is complex.

[0054] Finally, the system comprehensively analyzes the visible light features, thermal sensing features and pyroelectric features, and determines the recognition result through a machine learning model (such as a random forest), i.e., whether the to-be-identified image contains the target object (flame).

[0055] Optionally, in the embodiments of the present application, the recognition result refers to the judgment made by the system based on the comprehensive analysis of the visible light features, thermal sensing features and pyroelectric features. The specific form is a binary flag or a category label, indicating whether the target object is contained in the image to be recognized. In this embodiment, the target object specifically refers to the flame, that is, the system aims to judge whether there is a flame or a sign of fire source activity in the image.

[0056] In other words, the recognition result reflects the detection state of the target object (flame) in the image by the algorithm. If the judgment after feature fusion believes that there is a flame in the image, the recognition result will be marked as "exists" or "1"; otherwise, if it is judged that there is no flame in the image, the recognition result will be marked as "non-existent" or "0".

[0057] By the embodiments of the present application, the to-be-recognized region is determined according to the pixel probability of each pixel point in the to-be-recognized image, wherein the pixel probability of the pixel point is determined by the first chroma component and the second chroma component of the pixel point, and the pixel probability of the pixel point in the to-be-recognized region satisfies a preset probability discrimination condition; the visible light feature extraction operation and the thermal sensing feature extraction operation are respectively performed on the to-be-recognized region to obtain the visible light feature and the thermal sensing feature of the to-be-recognized region, wherein the visible light feature includes the circularity feature, the texture feature and the high-frequency energy feature, and the thermal sensing feature includes the temperature distribution feature and the centroid feature; and the pyroelectric feature extraction operation is performed on the to-be-recognized image to obtain the pyroelectric feature of the to-be-recognized region; the recognition result is determined according to the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the recognition result is used to represent whether the target object is contained in the to-be-recognized image, through multi-sensor information fusion and deep feature learning, the purpose of improving the object recognition accuracy is achieved, thereby realizing the technical effect of stable and efficient recognition of the target in a complex environment, and effectively solving the technical problem of low accuracy in object recognition by traditional single feature or sensor method.

[0058] As an optional solution, the determination of the recognition result according to the visible light feature, the thermal sensing feature and the pyroelectric feature includes: fusing the visible light feature, the thermal sensing feature and the pyroelectric feature to obtain a target feature vector; and determining the recognition result according to the target feature vector, wherein the recognition result is used to represent whether the flame is contained in the to-be-recognized image, and the target object includes the flame.

[0059] Optionally, in the embodiments of the present application, the target feature vector refers to the multi-dimensional data representation after fusing the visible light feature, the thermal sensing feature and the pyroelectric feature, including but not limited to circularity, texture, high-frequency energy, temperature distribution, centroid position and radiation intensity information of three infrared wave bands.

[0060] It should be noted that the feature fusion mode can be diversified, including but not limited to weighted average, deep learning model fusion, probability and statistics fusion, and the specific fusion method will be selected according to the characteristics of the target object, the requirements of the application scene and the performance requirements of the algorithm, and the present application does not limit this.

[0061] Exemplarily, the system first combines the circularity feature, texture feature and high-frequency energy feature obtained by preprocessing the to-be-identified image to form a visible light feature subset; at the same time, the temperature distribution feature and the center of mass feature obtained by the thermal imaging sensor constitute a thermal sensor feature subset; and the measurement values of the pyroelectric sensor at three wave bands form a pyroelectric feature subset. Subsequently, the three types of subsets are further integrated to generate a unified target feature vector through a selected fusion algorithm.

[0062] In an exemplary embodiment, taking a forest fire early warning system as an example, after receiving the to-be-identified image, the system first extracts the circularity, texture and high-frequency energy features by using color space conversion and statistical analysis; secondly, the temperature distribution and flame center information are obtained by the thermal imaging sensor; finally, the radiation intensity is measured by the pyroelectric sensor at the specified wave band, and these information is fused into a target feature vector. Subsequently, the system uses the trained random forest model, takes the target feature vector as the input, and outputs the recognition result to clearly indicate whether the image contains flame.

[0063] Through the embodiments of the present application, the multi-modal feature fusion is adopted to achieve the technical effect of high-precision and all-around recognition of flame, and the purpose of effectively detecting flame and improving the sensitivity and accuracy of the fire warning system is achieved even in a complex environment with poor light conditions or interference sources.

[0064] As an optional solution, the above-mentioned visible light feature extraction operation and thermal sensor feature extraction operation are respectively performed on the above-mentioned to-be-identified region to obtain the visible light feature and the thermal sensor feature of the above-mentioned to-be-identified region, including at least one of the following: obtaining the perimeter parameter and the area parameter of the above-mentioned to-be-identified region; determining the circularity feature according to the product of the area parameter and the perimeter parameter; generating a gray level co-occurrence matrix according to the above-mentioned to-be-identified region, and generating an energy parameter, an entropy parameter, a contrast parameter and a correlation parameter according to the above-mentioned gray level co-occurrence matrix; determining the texture feature based on the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter; performing a frequency domain decomposition operation on the above-mentioned to-be-identified region to obtain a plurality of subbands and a subband energy corresponding to each of the subbands, and determining the high-frequency energy feature based on the subband energy.

[0065] Optionally, in the embodiments of the present application, the region to be identified refers to an image region preliminarily screened by the system, which is likely to contain the target object (flame). The perimeter parameter and the area parameter represent the perimeter and the area size of the region, respectively. The circularity feature, the texture feature and the high-frequency energy feature constitute the visible light feature set. The circularity feature is calculated by the mathematical relationship between the area parameter and the perimeter parameter, and is used to describe whether the shape of the region is close to the typical irregular shape of the flame. The texture feature is determined based on the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter generated by the gray level co-occurrence matrix, which reflect the texture details and the visual complexity of the flame region. The high-frequency energy feature is extracted from the sub-band energy obtained by the frequency domain decomposition operation, and is used to capture the high-frequency visual features specific to the flame.

[0066] Optionally, in the embodiments of the present application, the thermal sensing features include the temperature distribution feature and the centroid feature. The temperature distribution feature is obtained by the thermal imaging sensor, and reflects the temperature distribution of the flame region. The centroid feature is calculated based on the temperature distribution of the flame region, and is used to locate the center position of the flame, thereby enhancing the accuracy of the identification.

[0067] It should be noted that the frequency domain decomposition operation can be implemented by various algorithms, including but not limited to wavelet transform, Fourier transform, etc. The specific selection depends on the system performance, the computing resources and the extraction requirements of the target features, and the present application does not limit the same. Similarly, the generation of the gray level co-occurrence matrix and the calculation of the parameters can also vary depending on the specific embodiments, such as adjusting the direction, distance or calculating the corresponding statistics of the matrix. These differences in details do not constitute a limitation on the technical solutions of the present application.

[0068] Exemplarily, in the preprocessing stage, the system performs color space conversion on the image to facilitate feature extraction. Subsequently, the system determines the circularity feature by calculating the perimeter and the area of the specific region, obtains the texture feature by constructing the gray level co-occurrence matrix and calculating the energy, the entropy, the contrast and the correlation of the matrix, extracts the high-frequency sub-band energy by decomposing the image through wavelet transform to determine the high-frequency energy feature, and calculates the centroid feature based on the temperature data obtained by the thermal imaging sensor to supplement the thermal sensing feature set.

[0069] In an exemplary embodiment, taking a fire monitoring system as an example, the image captured by the camera is first processed, and the system screens out the suspected flame region according to the color space model. Then, the system calculates the circularity feature of the region, and constructs the gray level co-occurrence matrix to analyze the statistical parameters of the matrix to determine the texture feature. Next, the high-frequency energy feature of the flame is identified through frequency domain analysis. The temperature distribution data provided by the thermal imaging sensor and the centroid calculation constitute the thermal sensing feature. Finally, all the features are integrated, and the machine learning model is used to determine whether the region contains the flame.

[0070] Through the embodiments of the present application, multi-modal feature fusion analysis is adopted to realize accurate identification and positioning of objects, and the purpose of improving object identification sensitivity and accuracy is achieved, especially the identification capability in complex environments is enhanced, and false positives and false negatives are reduced.

[0071] As an optional solution, the above-mentioned determination of the above-mentioned circularity feature according to the product parameter of the above-mentioned area parameter and the above-mentioned perimeter parameter comprises: determining the product of a preset parameter and the product parameter and the square of the perimeter parameter as the value of the circularity feature, wherein the preset parameter is a positive integer greater than 0.

[0072] Optionally, in the embodiments of the present application, the value of the circularity feature refers to a numerical value describing the regularity of the shape of the region obtained by mathematical operation processing of the area parameter and the perimeter parameter. The preset parameter acts as an adjustment factor in this process, balancing the influence of area and perimeter on the circularity feature, ensuring that the circularity feature can accurately reflect the shape characteristics of the target object. The selection of the preset parameter includes but is not limited to 4π or other positive integers greater than 0, and the specific value can be adjusted according to actual application requirements and target shape characteristics to optimize the identification performance.

[0073] It should be noted that the specific formula for calculating the circularity feature can be expressed as:

[0074]

[0075] Where S is the area of the region A(x, y), and L is the perimeter of the region A(x, y).

[0076] In addition, if the multiple pixel points in the region to be identified are discrete and cannot form a closed region, the calculation of the circularity feature can be: obtaining the sum of the Euclidean distances between each adjacent pixel point in counterclockwise order (or clockwise) to obtain the perimeter parameter, and the area parameter can be the sum of the areas occupied by each of the multiple pixel points.

[0077] Through the embodiments of the present application, multi-modal feature fusion analysis is adopted to realize accurate identification and positioning of objects, and the purpose of improving object identification sensitivity and accuracy is achieved, especially the identification capability in complex environments is enhanced, and false positives and false negatives are reduced.

[0078] As an optional solution, the generating the gray level co-occurrence matrix according to the to-be-identified region comprises: obtaining a plurality of preset matrix direction parameters; and generating a plurality of target gray level co-occurrence matrices according to the plurality of matrix direction parameters and a preset distance parameter, wherein the gray level co-occurrence matrix comprises the plurality of target gray level co-occurrence matrices, and one of the matrix direction parameters corresponds to one of the target gray level co-occurrence matrices.

[0079] Optionally, in the embodiments of the present application, the gray level co-occurrence matrix refers to a statistical matrix formed by calculating the joint probability distribution of the pixel gray level values in a to-be-identified region in a specific direction and distance. The matrix direction parameter and the distance parameter are key parameters for defining the co-occurrence matrix, and are used to capture different texture information in the image. One matrix direction parameter corresponds to one target gray level co-occurrence matrix, and includes but is not limited to 0 degrees, 45 degrees, 90 degrees and 135 degrees, and a distance parameter of 1 to 5 pixels. The specific value selection depends on the characteristics of the target object and the application scenario requirements.

[0080] It should be noted that the process of obtaining the gray level co-occurrence matrix involves selecting a plurality of direction and distance parameters, and the setting of these parameters can enhance the system's ability to capture texture features, and is suitable for texture analysis of different scales and directions.

[0081] By adjusting the direction parameter and the distance parameter, the texture structure of the target region can be more comprehensively described, and the description of the features is improved. The present application does not limit the setting of the specific parameters.

[0082] Exemplarily, a plurality of direction parameters and a distance parameter are first set, and then according to these parameters, a corresponding gray level co-occurrence matrix is generated for the to-be-identified region in each direction. The plurality of target gray level co-occurrence matrices generated in this process can capture and quantify the texture features of the to-be-identified region from multiple angles.

[0083] In an exemplary embodiment, taking an industrial fire monitoring system as an example, after the system receives a to-be-identified image, a suspected flame region is first determined based on color space analysis. Then, by setting a plurality of direction parameters (for example, 0 degrees, 45 degrees, 90 degrees and 135 degrees) and distance parameters (for example, 1 to 5 pixels), the system generates a plurality of target gray level co-occurrence matrices, which respectively correspond to different texture directions and scales. These matrices are used to in-depth analyze the texture details of the region, and provide key texture information for subsequent feature fusion and flame recognition.

[0084] Taking 0 degrees as an example, the system will calculate the gray co-occurrence relationship of each pixel point in the to-be-identified region with adjacent pixel points in the 0-degree direction and a preset distance parameter (for example, 1-5 pixels). This process constructs the target gray co-occurrence matrix in the 0-degree direction by traversing all the pixels in the image region and counting the combination frequency of the gray values in the given direction and distance, so as to construct the target gray co-occurrence matrix in the 0-degree direction. Each element in the matrix represents the occurrence frequency of the corresponding gray level combination in the image region. This matrix can finely describe the texture features of the image in a specific direction, including the gray change mode and distribution between pixels, and provides a basis for the calculation of subsequent texture feature extraction such as energy parameter, entropy parameter, contrast parameter and correlation parameter, thereby providing key support for the accuracy of flame recognition.

[0085] Taking 0 degrees as an example, the specific operation is as follows:

[0086] S1, determine the 0-degree direction and the distance parameter, for example, set the distance parameter to 1 pixel.

[0087] S2, for each pixel in the to-be-identified region, find the adjacent pixel in the 0-degree direction with a distance of 1 pixel.

[0088] S3, count the occurrence frequency of the gray value combination of the two pixels in the image, and construct the gray co-occurrence matrix.

[0089] S4, traverse the entire region to ensure that all pixels participate in the co-occurrence relationship statistics.

[0090] S5, according to the constructed co-occurrence matrix, extract the energy parameter, entropy parameter, contrast parameter and correlation parameter to provide quantitative indexes for the description of the texture features.

[0091] Through the embodiments of the present application, the multi-direction gray co-occurrence matrix feature extraction technology is adopted to realize comprehensive capture and quantization of the texture features of the target region, so as to accurately describe and recognize the texture structure of the target object (such as flame) under different environments and light conditions, thereby achieving the purpose of improving the recognition accuracy.

[0092] As an optional solution, the method further comprises: generating the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter corresponding to each target gray level co-occurrence matrix respectively, wherein one target gray level co-occurrence matrix corresponds to one set of target parameters, and the set of target parameters comprises the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter; determining an energy mean value parameter and an energy standard deviation parameter according to the energy parameter in each set of target parameters; determining an entropy mean value parameter and an entropy standard deviation parameter according to the entropy parameter in each set of target parameters; determining a contrast mean value parameter and a contrast standard deviation parameter according to the contrast parameter in each set of target parameters; determining a correlation mean value parameter and a correlation standard deviation parameter according to the correlation parameter in each set of target parameters; and determining the value of the texture feature using the energy mean value parameter and the energy standard deviation parameter, the entropy mean value parameter and the entropy standard deviation parameter, the contrast mean value parameter and the contrast standard deviation parameter, and the correlation mean value parameter and the correlation standard deviation parameter.

[0093] Optionally, in the embodiments of the present application, the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter respectively refer to the statistical feature values calculated from each target gray level co-occurrence matrix, which are used to describe the uniformity, complexity, contrast intensity and correlation between gray values of the image region. One set of target parameters contains the four parameters extracted from the target gray level co-occurrence matrix of a specific direction. By calculating the mean value and the standard deviation in each set of target parameters, the system can obtain the energy mean value parameter, the energy standard deviation parameter, the entropy mean value parameter, the entropy standard deviation parameter, the contrast mean value parameter, the contrast standard deviation parameter, the correlation mean value parameter and the correlation standard deviation parameter, which are finally used to determine the value of the texture feature, including but not limited to the uniformity, complexity, contrast and correlation of the texture.

[0094] It should be noted that there are many methods for calculating the target parameters, such as directly calculating from the co-occurrence matrix or indirectly deriving through normalization of the matrix. In addition, the calculation standard and range of the parameters can also be adjusted according to the specific application scene and data processing requirements, such as using different statistics or calculation methods. The implementation of the present application is not limited to a specific calculation method or parameter setting, and these details can be flexibly adjusted according to actual needs, which are not limited by the present application.

[0095] Exemplarily, the system first calculates the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter corresponding to each direction respectively according to the target gray level co-occurrence matrix of multiple directions, to form multiple groups of target parameters. Then, the system further analyzes each group of target parameters to determine their mean value and standard deviation, so as to comprehensively consider the statistical characteristics of the texture features of different directions. Finally, the statistical characteristics are integrated as the value of the texture feature, which is used for subsequent feature fusion and flame recognition.

[0096] In an exemplary embodiment, taking the intelligent security monitoring system as an example, after identifying the suspected flame region in the received image to be analyzed, the system first extracts the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter based on the gray level co-occurrence matrix of multiple directions. Then, the system calculates the mean value and the standard deviation of these parameters, i.e., the energy mean value parameter, the energy standard deviation parameter, the entropy mean value parameter, the entropy standard deviation parameter, the contrast mean value parameter, the contrast standard deviation parameter, the correlation mean value parameter and the correlation standard deviation parameter.

[0097] Through the embodiments of the present application, the calculation and integration of the statistical parameters of the multi-direction co-occurrence matrix are adopted to realize the fine description of the texture features of the region to be recognized, so as to accurately capture and quantify the texture details of the image region in a complex environment, thereby improving the object recognition accuracy and effectively overcoming the limitations of single texture feature description.

[0098] As an optional solution, the generation of the gray level co-occurrence matrix according to the region to be recognized, the generation of the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter according to the gray level co-occurrence matrix includes: accumulating and summing the square of each element in the gray level co-occurrence matrix to determine the energy parameter; accumulating and summing the negative value of the product of the value of each element in the gray level co-occurrence matrix and the logarithm value of the value to determine the entropy parameter; accumulating and summing the product of the value of each element in the gray level co-occurrence matrix and the square of the difference between the row index and the column index of the corresponding element position to determine the contrast parameter; subtracting the product of the mean value of the row index and the mean value of the column index from the sum value of the product of the value of each element in the gray level co-occurrence matrix and the row index and the column index of the corresponding element position, and then dividing by the product of the standard deviation of the row index and the standard deviation of the column index to determine the correlation parameter.

[0099] Optionally, in the embodiments of the present application, the energy parameter is calculated by accumulating the square value of all elements in the gray level co-occurrence matrix, which measures the uniformity and repeatability of the texture in the image;

[0100] Optionally, in the embodiments of the present application, the entropy parameter is determined by accumulating the product of each element value and its natural logarithm and then taking the negative value, which reflects the complexity and diversity of the image texture;

[0101] Optionally, in the embodiments of the present application, the contrast parameter is calculated according to the sum of the product of each element value and the square of the difference between the corresponding position row index and column index, which describes the contrast and sharpness of the texture.

[0102] Optionally, in the embodiments of the present application, the correlation parameter quantifies the correlation degree between the elements in the gray level co-occurrence matrix, including but not limited to the change distribution of the gray level and the structural correlation in the image region.

[0103] It should be noted that although the mathematical formulas of the energy, entropy, contrast and correlation parameters of the gray level co-occurrence matrix and its statistical parameters are specifically described in the present application, in actual application, the details of these formulas or other statistical quantities such as offset, angular second moment, etc. can be adjusted according to different image features, recognition accuracy requirements or calculation efficiency considerations to optimize the description ability of the texture features. In addition, the construction of the gray level co-occurrence matrix can also be based on various gray levels, such as 256 levels or higher, which is not limited in the present application.

[0104] Exemplarily, the system first generates a gray level co-occurrence matrix for the region to be identified, and then calculates the energy parameter, entropy parameter, contrast parameter and correlation parameter of each matrix according to the formulas defined above. This process includes but is not limited to: the square value of the element is accumulated and summed to determine the energy parameter, the product of the element value and the natural logarithm is accumulated and then negated to determine the entropy parameter, the sum of the product of the element value and the square of the difference between the row index and the column index to determine the contrast parameter, and the operation based on the product of the element value and the row and column index to determine the correlation parameter, thereby constructing a set of parameter collection which comprehensively describes the texture features of the image.

[0105] In an exemplary embodiment, ASM represents the energy parameter, ENT represents the entropy parameter, CON represents the contrast parameter, and COR represents the correlation parameter:

[0106]

[0107] wherein μ n , μ m are the mean values in x and y directions respectively, σ n , σ m are the standard deviations in x and y directions respectively, and P(n, m) represents the element in the gray level co-occurrence matrix, i.e. the probability of the pixels with gray values of n and m appearing simultaneously in a specific direction and distance in the image.

[0108] Then, the texture features F GLCM calculated by the gray level co-occurrence matrix can be represented as:

[0109] F GLCM = [μ ASM , σ ASM , μENT , σ ENT , μ CON , σ CON , μ COR , σ COR ]

[0110] wherein μ ASM represents an energy mean parameter, σ ASM represents an energy standard deviation parameter, μ ENT represents an entropy mean parameter, σ ENT represents an entropy standard deviation parameter, μ CON represents a contrast mean parameter, σ CON represents a contrast standard deviation parameter, μ COR represents a correlation mean parameter, σ COR represents a correlation standard deviation parameter.

[0111] In an exemplary embodiment, taking the intelligent fire warning system as an example, after capturing the image area suspected of fire, the system first generates a gray level co-occurrence matrix. Then, according to each element of the matrix, the system calculates the energy parameter, the entropy parameter, the contrast parameter and the correlation parameter, which respectively reflect the uniformity, complexity, contrast intensity and correlation between gray values of the texture of the image area. Through this series of statistical analysis, the system can more deeply understand the texture details of the image area, providing high-quality texture feature data for subsequent feature fusion and flame detection, ensuring the recognition accuracy and robustness under various environmental conditions.

[0112] Through the embodiments of the present application, the multi-dimensional statistical feature extraction method of the gray level co-occurrence matrix is adopted to realize the fine description of the texture characteristics of the image area, achieve the goal of effectively recognizing and distinguishing fire and other similar textures in complex backgrounds, and enhance the overall performance of the flame detection system. This strategy improves the perception and analysis ability of the system for texture details by comprehensively considering the texture information of the image area from different angles, so as to more accurately serve the fire monitoring task.

[0113] As an optional solution, the above-mentioned performing frequency domain decomposition operation on the above-mentioned to-be-recognized region to obtain a plurality of subbands and a subband energy corresponding to each of the subbands, and determining the high-frequency energy feature based on the subband energy, includes: obtaining the subband energy of a high-frequency subband in the plurality of subbands; determining the ratio of the subband energy of the high-frequency subband to the subband energy of each of the subbands as the value of the high-frequency energy feature.

[0114] Optionally, in the embodiments of the present application, the frequency domain decomposition operation refers to the process of converting the image signal into the frequency domain for decomposition, which is usually completed through Discrete Wavelet Transform (DWT) or similar frequency domain analysis technology; the sub-band refers to the image signal part of different frequency ranges after frequency domain decomposition; the sub-band energy is represented as the intensity or activity of each sub-band signal, which can be obtained by calculating the square sum of the pixel values in the sub-band and then taking the square root; the ratio of the sub-band energy of the high frequency sub-band to the total energy of all sub-bands constitutes the value of the high frequency energy feature, which is used to represent the proportion of high frequency components in the image, including but not limited to the density of edge, detail and other texture information.

[0115] It should be noted that the frequency domain decomposition operation can use different types of wavelet basis functions, such as Haar wavelet, Daubechies wavelet or Symlets wavelet, and the decomposition level and sub-band division method are flexible, and the specific selection depends on the capture requirements of image features and the constraints of computing resources.

[0116] In addition, the calculation method of sub-band energy can use statistical quantities such as average value, maximum value or kurtosis in addition to the square root method of the square sum of pixel values mentioned above, which is not limited in the present application, and can be adjusted according to the actual situation in specific implementation.

[0117] For example, after receiving the to-be-identified region, the discrete wavelet transform is applied to decompose it into different sub-bands, including a low frequency sub-band and a plurality of high frequency sub-bands; then the sub-band energy of each sub-band is calculated, i.e. the square root of the square sum of the pixel values in each sub-band; then the sub-band energy of the high frequency sub-band is selected, and the ratio of their sum to the sum of all sub-band energies is calculated, which is the value of the high frequency energy feature, which is used for subsequent feature analysis or fusion.

[0118] In an exemplary embodiment, taking an industrial fire monitoring system as an example, the system identifies a suspected flame region in the received image and performs frequency domain decomposition processing on it. First, the Daubechies wavelet basis function is used to perform two-layer decomposition on the region, obtaining four sub-bands:

[0119] The horizontal direction high frequency sub-band (cH), the vertical direction high frequency sub-band (cV), the diagonal direction high frequency sub-band (cD) and the low frequency sub-band (cA).

[0120] Then, the system calculates the sub-band energy of the four sub-bands, wherein the sub-band energy of cH, cV and cD is regarded as the high frequency sub-band energy. Next, the system calculates the proportion of the sum of the high frequency sub-band energy to the sum of all sub-band energies, which constitutes the value of the high frequency energy feature, as follows:

[0121] First, the Daubechies-4 wavelet basis is used to perform 3-layer decomposition on the to-be-recognized region to obtain subbands {cA3, cH3, cV3, cD3, cH2, cV2, cD2, cH1, cV1, cD1}, and the energy of each subband can be represented as

[0122]

[0123] For the three-layer-decomposed image, the total energy is The energy of the flame image is concentrated in the high-frequency subbands, i.e., the 1st to 2nd layers of cH, cV and cD, and the high-frequency energy feature can be represented as:

[0124]

[0125] Through the embodiments of the present application, the frequency domain analysis technology based on the high-frequency subband energy feature is adopted to realize accurate quantification of the details and edge information in the image, achieve effective recognition of the flame region in a complex image background, and reduce the false alarm rate. By focusing on the energy information of the high-frequency subbands, the capture of the unique texture features of the flame is strengthened, so that the system can still maintain high flame detection performance under adverse conditions such as light changes and background noise interference.

[0126] As an optional solution, the above-mentioned performing a thermal sensing feature extraction operation on the to-be-recognized region to obtain a thermal sensing feature includes: acquiring the temperature of each pixel point in the to-be-recognized region using a thermal imaging module; determining a temperature distribution feature according to a temperature mean value parameter, a temperature variance parameter and a temperature scale parameter of the pixel points; determining a target pixel point set satisfying a preset temperature condition according to the temperature of each pixel point, and acquiring a mass point pixel point of the to-be-recognized region, and determining the centroid feature according to the mass point pixel point and the target pixel point set.

[0127] Optionally, in the embodiments of the present application, the frequency domain decomposition operation refers to the process of converting the image signal into different frequency subbands by using frequency domain analysis techniques such as wavelet transform or Fourier transform; the subband energy is represented as the total intensity of each subband signal, which can be calculated by the sum of squares of pixel values within the subband or similar statistics; the value of the high-frequency energy feature is determined based on the ratio of the high-frequency subband energy to the total energy of all subbands, reflecting the relative proportion of high-frequency components in the image; the temperature mean parameter, temperature variance parameter and temperature scale parameter of the pixel point are used to construct the temperature distribution feature, which quantifies the average level, fluctuation degree and distribution form of the regional temperature respectively; the target pixel point set refers to the pixel set whose temperature meets the preset condition (such as being higher than the threshold), which is used for subsequent calculation of the centroid feature; the mass pixel point refers to the weight parameter in the centroid calculation, which is related to the temperature value of the pixel point; the determination of the centroid feature combines the mass pixel point and the target pixel point set, and is completed by calculating the weighted centroid position, which is used to describe the concentration degree of the temperature distribution, including but not limited to the positioning of the flame center.

[0128] It should be noted that there are many techniques for frequency domain decomposition, and wavelet transform and Fourier transform are optional embodiments in the present application. The specific transformation type and decomposition level can be adjusted according to the actual application scenario and computing resources. The calculation of temperature parameters also has certain flexibility, and details such as the quantization precision of pixel temperature and the statistical method of temperature distribution feature can be customized as needed, which is not limited in the present application.

[0129] Exemplarily, the system performs frequency domain decomposition and temperature distribution feature extraction on the region to be identified. First, wavelet transform frequency domain decomposition is performed to obtain multiple subbands including low frequency and high frequency; second, the ratio of the sum of high-frequency subband energy to the sum of all subband energy is calculated to determine the value of high-frequency energy feature; at the same time, the system obtains the temperature value of each pixel point through the thermal imaging module, and calculates the temperature mean, variance and scale based on these temperature values to form the temperature distribution feature; then, the target pixel point set whose temperature is higher than the preset threshold is selected; finally, the centroid feature, i.e. the weighted centroid position, is calculated by combining the temperature value and position information of the target pixel point.

[0130] In an exemplary embodiment, taking a forest fire monitoring system as an example, after receiving an image of a suspected flame area, the system first acquires temperature information of each pixel point using a thermal imaging module, calculates a temperature mean parameter, a temperature variance parameter and a temperature scale parameter based on the temperature values, and uses the parameters to construct a temperature distribution feature. Next, the system identifies pixel points with a temperature higher than a preset threshold to form a target pixel point set. At the same time, the system performs wavelet transform frequency domain decomposition on the image, acquires subbands of different frequency ranges, and calculates the energy of the high-frequency subbands, and determines the ratio of the energy of the high-frequency subbands to the energy of all subbands as a high-frequency energy feature. Finally, the system determines a mass point pixel based on the target pixel point set and the corresponding temperature values, and calculates the position of the mass center using a weighted centroid algorithm. The position reflects the concentration of the high-temperature area and is added to the feature vector of the flame detection as a centroid feature.

[0131] By the embodiments of the present application, the analysis method combining frequency domain decomposition and temperature distribution statistics is adopted to comprehensively consider the high-frequency components and temperature distribution characteristics in the image area, so that the flame can be accurately identified in a complex natural environment, and the false positive rate is reduced.

[0132] As an optional solution, the determination of the centroid feature based on the mass point pixel and the target pixel point set includes: in a case where the mass point pixel belongs to the target pixel point set, the value of the centroid feature is determined as a positive integer 1; and in a case where the mass point pixel does not belong to the target pixel point set, the value of the centroid feature is determined as 0.

[0133] Optionally, in the embodiments of the present application, the mass point pixel refers to a mass center position pixel calculated based on the temperature values of the pixel points in the image area; the target pixel point set refers to a set of pixel points in the image whose temperature satisfies a preset condition (such as being higher than a certain threshold); and the value of the centroid feature is determined based on whether the mass point pixel is located in the target pixel point set. If yes, the value is a positive integer 1; otherwise, the value is 0.

[0134] It should be noted that the specific setting of the preset condition, such as the selection of the temperature threshold, can be flexibly adjusted according to different application scenarios and environmental backgrounds. Similarly, the value logic of the centroid feature can also be extended according to actual needs, for example, using continuous values or multiple integer values of different levels to express the relative relationship between the mass center and the high-temperature area, which is not limited in the present application.

[0135] Exemplarily, feature extraction is performed on the to-be-identified region. First, the temperature value of each pixel point in the image is obtained through the thermal imaging module to form a temperature image. Then, all pixel points with a temperature higher than a preset threshold are extracted to form a target pixel point set. Then, the temperature centroid of the target pixel point set, i.e., the mass point pixel point of the image, is calculated according to the temperature value and position coordinates of each pixel point in the target pixel point set. Finally, it is checked whether the mass point pixel point belongs to the target pixel point set, and the value of the centroid feature, i.e., 1 or 0, is determined based on this to serve as an important parameter for flame detection.

[0136] In an exemplary embodiment, taking a city high-rise building fire monitoring system as an example, after receiving the visible light image and the thermal image of the suspected flame region, the system first extracts the temperature information in the image using thermal imaging technology to form a temperature image. Then, the system identifies all pixel points with a temperature exceeding a preset threshold and groups these pixel points into a target pixel point set. Next, the system calculates the temperature centroid of the target pixel point set, i.e., the mass point pixel point. Finally, if the mass point pixel point is located within the target pixel point set, it indicates that the high-temperature region is concentrated and there may be a flame, and in this case, the system sets the value of the centroid feature to 1. Conversely, if the mass point pixel point is not covered by the target pixel point set, it is considered that the region is not a flame, and in this case, the system sets the value of the centroid feature to 0. This process ensures accurate identification of the flame by the system, and even in a complex urban environment, it can effectively distinguish between the true flame signal and potential interference sources.

[0137] As an optional solution, the above-mentioned performing pyroelectric feature extraction on the to-be-identified image to obtain the pyroelectric feature of the to-be-identified region includes: performing pyroelectric feature extraction on the to-be-identified image using a first waveband filter, a second waveband filter, and a third waveband filter to obtain a first pyroelectric feature, a second pyroelectric feature, and a third pyroelectric feature, wherein the first waveband filter, the second waveband filter, and the third waveband filter are all different; and determining the pyroelectric feature using the first pyroelectric feature, the second pyroelectric feature, and the third pyroelectric feature.

[0138] Optionally, in the embodiments of the present application, the pyroelectric feature extraction operation refers to a process of detecting and analyzing infrared radiation in the image using a pyroelectric sensor combined with a waveband filter; the first waveband filter, the second waveband filter, and the third waveband filter are respectively used to filter infrared rays in different wavelength ranges, so that the pyroelectric sensor can receive radiation signals of specific wavebands; and the pyroelectric feature is determined by analyzing the signal strength obtained by the sensor under different wavebands, including but not limited to radiation intensity, peak frequency, or signal waveform features, etc.

[0139] It should be noted that the specific wavelength range of the waveband filter can be adjusted according to the infrared radiation characteristics of the target detection object, for example, the first waveband filter can be for a wavelength of about 3.8 microns, the second waveband filter can focus on radiation near 4.4 microns, and the third waveband filter can detect signals in the 5.3 micron waveband; there are various implementation schemes for determining the pyroelectric feature, such as directly using the signal strength as the feature, or further analyzing the time sequence characteristics of the signal, such as periodicity, pulse width, etc., which are not limited by the present application, as long as the pyroelectric feature can effectively reflect the infrared radiation difference of the region to be identified.

[0140] Exemplarily, the pyroelectric feature extraction is performed on the above-mentioned region to be identified, first, the infrared radiation of non-target wavelength is filtered out through the first waveband filter, then the infrared signal strength under the first waveband is received and measured using the pyroelectric sensor, to obtain the first pyroelectric feature; then, the system replaces the second waveband filter to perform similar operations to obtain the second pyroelectric feature; finally, the third waveband filter is used to perform feature extraction to obtain the third pyroelectric feature; the set of all pyroelectric features constitutes the pyroelectric feature of the region to be identified, which is used for subsequent flame identification or classification.

[0141] In an exemplary embodiment, taking the intelligent kitchen fire warning system as an example, after capturing the image of the suspected flame, the integrated pyroelectric sensor module is activated, which is equipped with three different waveband filters. First, the sensor captures the infrared radiation signal in the 3.8 micron waveband through the first waveband filter, analyzes the signal strength, and obtains the first pyroelectric feature; then, the second waveband filter (for example, 4.4 microns) is replaced, and the infrared signal strength is measured and recorded again to obtain the second pyroelectric feature; finally, the third waveband filter (for example, 5.3 microns) is used to complete the last signal acquisition and determine the third pyroelectric feature. By comparing and analyzing the three features, the system can more accurately determine whether the suspected flame region really exists flame activity according to the radiation characteristics of the flame in different wavebands, thereby improving the accuracy and response speed of the fire warning.

[0142] Through the embodiments of the present application, the infrared radiation detection technology combining multi-waveband pyroelectric sensors with waveband filters is adopted to realize all-around detection of the infrared radiation characteristics of the suspected flame region, achieving the purpose of improving the sensitivity and specificity of flame detection. Especially in a harsh environment with low light or smoke obstruction, the limitations of visual and thermal imaging sensors can be effectively overcome, ensuring the reliability and timeliness of flame detection.

[0143] In an exemplary embodiment, taking the identification of the target object as a flame as an example, applied to the field of fire safety monitoring, the flow chart is as shown in Figure 3

[0144] ​Step one: First, convert the image from RGB color space to YCbCr color space, where Y represents the luminance component of the channel, Cb represents the blue chrominance component of the channel, and Cr represents the red chrominance component of the channel. Flame pixels in the (Cr, Cb) space basically conform to a two-dimensional normal distribution, so we can get

[0145]

[0146] where, S Cr , S Cb are the mean and standard deviation of Cr and Cb, respectively.

[0147] Next, the probability f(Cr, Cb) of each pixel point, i.e., the probability of the flame pixel, is calculated. When f(Cr, Cb) is greater than a threshold T, the coordinates of the pixel are recorded. Finally, the flame region A(x, y) is obtained. In this patent, T = 3e-15.

[0148] Step two: The value of flame circularity is usually low, and this feature can accurately exclude light, sunlight, and other interference sources with high circularity. The flame circularity feature F C is calculated for the flame region A(x, y) extracted in step one, and the formula is as follows:

[0149]

[0150] where S is the area of the region A(x, y), and L is the perimeter of the region A(x, y).

[0151] Step three: Texture information can accurately reflect the difference between the flame and other objects. The gray level co-occurrence matrix can reflect the texture features of the image at multiple angles. The mean and standard deviation of energy (ASM), entropy (ENT), contrast (CON), and correlation (COR) are used to extract flame features in the embodiments of the present application. Energy (ASM) can describe the uniformity and fineness of image texture, and the larger the value, the more regular the image texture distribution. Entropy (ENT) can describe the complexity of image texture, and the larger the value, the more obvious the texture groove, and the clearer the image. Contrast (CON) can describe the clarity of the image and the depth of the image texture, and the larger the value, the clearer the image. Correlation (COR) describes the correlation of the gray level co-occurrence matrix, and the larger the value, the more gentle the gradient of the matrix elements.

[0152] First, construct a gray level co-occurrence matrix for the extracted suspected flame region A(x, y), then normalize the gray level co-occurrence matrix to obtain the matrix P(x, y), and then the energy (ASM), entropy (ENT), contrast (CON), and correlation (COR) can be represented as:

[0153]

[0154]

[0155] where μ n , μ m are the mean values of x, y direction, σ n , σ m are the standard deviations of x, y direction.

[0156] Then, the flame texture feature F GLCM calculated by the gray level co-occurrence matrix can be expressed as:

[0157] F GLCM = [μ ASM , σ ASM , μ ENT , σ ENT , μ CON , σ CON , μ COR , σ COR ]

[0158] where μ ASM , σ ASM , μ ENT , σ ENT , μ CON , σ CON , μ COR , σ COR are the mean values and standard deviations of energy (ASM), entropy (ENT), contrast (CON) and correlation (COR).

[0159] Step four: first, the suspected flame region is decomposed into 3 layers using Daubechies-4 wavelet basis, and the sub-band {cA3, cH3, cV3, cD3, cH2, cV2, cD2, cH1, cV1, cD1} is obtained, and the energy of each sub-band can be expressed as:

[0160]

[0161] For the image decomposed into three layers, the total energy E is: The energy of the flame image is concentrated in the high-frequency sub-band, i.e. the 1st to 2nd layers of cH, cV and cD, and the high-frequency energy feature can be expressed as:

[0162]

[0163] Step five: first, the temperature of region A(x, y) obtained in step one is acquired by the thermal imaging module to obtain T(x, y), then the histogram fitting of the temperature T(x, y) of all pixel points is performed using the generalized Gaussian distribution to obtain three fitting parameters, respectively, the mean, variance and scale of the temperature of all pixels in region A, and finally the temperature distribution feature F is obtained T = [T μ , T θ , T φ ];

[0164] Step six: first, mark the top 10% high temperature points in region A(x, y) as region A 10％ (x, y), then calculate the centroid of the flame, as shown in Figure 4 , the centroid calculation formula is:

[0165]

[0166] Wherein, the pixel coordinates of the flame region are (x i , y i ), wherein i is the pixel index, and I(x i , y i ) is the gray value of the pixel. Then determine whether the centroid coordinates (x ce , y ce ) belong to the top 10% high temperature point region A 10％ (x, y), if it belongs to, the centroid feature F ce = 1, otherwise F ce = 0.

[0167] Step seven: as shown in Figure 5 , first, install the pyroelectric sensor between the visible light lens and the thermal imaging lens, as shown in the following figure, and the field of view angle is greater than that of the visible light and thermal imaging lens. Three pyroelectric sensors use 3.8um, 4.4um and 5.3um bandpass filters respectively. For each frame of picture, the pyroelectric sensor separates the infrared spectrum at the same time to obtain the pyroelectric parameter feature F P .

[0168] Step eight: fuse all features of the three channels of the visible light channel, the thermal imaging channel and the pyroelectric sensor to obtain the final flame feature vector, denoted as F = [F C , F GLCM , F high , F T , F ce , F P ].

[0169] Step nine: use random forest technology, take F as the input of random forest technology, and output the determination result of the flame.

[0170] Through the embodiments of the present application, the flame is detected in combination with visible light images, thermal imaging temperature and three waveband three-dimensional data. In image feature extraction, not only the circularity feature and texture feature of the flame are extracted in the image domain, but also the image is converted to the frequency domain, and the flame detail feature is analyzed by analyzing the subband energy. At the same time, the infrared detection technology is combined with the flame temperature, the centroid and the waveband radiation characteristics. Through the fusion learning of multi-dimensional features, even when the scene is disturbed by light, temperature or other interference, the stability and accuracy of the system to identify the flame can be ensured. In combination with the temperature and the three waveband features, the sensitivity of the system to identify the flame can be improved. At the same time, for the invisible light flame or the flame with weak color, such as alcohol fire, methane fire, etc., the obvious flame features can be extracted, the flame can be accurately identified, the false negative rate can be reduced, and the fire risk can be reduced.

[0171] That is, the flame is identified in combination with visible light images, thermal imaging temperature and three waveband three-dimensional data, the accuracy of flame identification is improved by rich and comprehensive feature information, and the method has low complexity; the visible light image contains rich detail and texture information. The circularity feature and texture feature of the flame are first extracted on the visible light image, and the image is converted to the frequency domain, and the frequency domain subband energy feature is extracted by analyzing the subband energy. Considering that the flame temperature is also an obvious feature of the flame. The temperature distribution of the flame region is counted by the thermal imaging sensor, and the centroid position calculated is combined with the temperature feature to make a composite judgment, so that the accuracy of the centroid feature judgment is improved. Three pyroelectric sensors are used to extract the radiation characteristics of the flame waveband, and the accuracy of flame identification is improved.

[0172] It can be understood that in the specific embodiments of the present application, user information and other related data are involved, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0173] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0174] According to another aspect of the embodiments of the present application, an object recognition device for implementing the above object recognition method is also provided. As shown in the Figure 6 device includes:

[0175] The determining module 602 is used to determine the region to be identified based on the pixel probability of each pixel in the image to be identified, wherein the pixel probability of each pixel is determined by the first chromaticity component and the second chromaticity component of the pixel, and the pixel probability of the pixels in the region to be identified satisfies a preset probability discrimination condition.

[0176] The execution module 604 is used to perform visible light feature extraction and thermal sensing feature extraction operations on the region to be identified, respectively, to obtain the visible light features and thermal sensing features of the region to be identified. The visible light features include circularity features, texture features, and high-frequency energy features, and the thermal sensing features include temperature distribution features and centroid features. The module also performs pyroelectric feature extraction on the image to be identified to obtain the pyroelectric features of the region to be identified.

[0177] The identification module 606 is used to determine the identification result based on the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the identification result is used to indicate whether the image to be identified contains a target object.

[0178] As an optional approach, the aforementioned device is used to determine the recognition result based on visible light features, thermal sensing features, and pyroelectric features in the following manner: fusing visible light features, thermal sensing features, and pyroelectric features to obtain a target feature vector; and determining the recognition result based on the target feature vector, wherein the recognition result is used to indicate whether the image to be recognized contains a flame, and wherein the target object includes a flame.

[0179] As an optional solution, the above-mentioned device is used to perform visible light feature extraction and thermal sensing feature extraction operations on the region to be identified in the following manner to obtain the visible light features and thermal sensing features of the region to be identified, including at least one of the following: obtaining the perimeter parameter and area parameter of the region to be identified; determining the circularity feature based on the product of the area parameter and the perimeter parameter; generating a gray-level co-occurrence matrix based on the region to be identified, and generating energy parameter, entropy parameter, contrast parameter and correlation parameter based on the gray-level co-occurrence matrix; determining texture features based on the energy parameter, entropy parameter, contrast parameter and correlation parameter; performing a frequency domain decomposition operation on the region to be identified to obtain multiple sub-bands and the sub-band energy corresponding to each sub-band, and determining high-frequency energy features based on the sub-band energy.

[0180] As an optional solution, the above-mentioned device is used to determine the roundness feature based on the product parameter of the area parameter and the perimeter parameter in the following manner: the product of the preset parameter and the product parameter and the square of the perimeter parameter are determined as the value of the roundness feature, wherein the preset parameter is a positive integer greater than 0.

[0181] As an optional solution, the apparatus is further configured to generate the gray level co-occurrence matrix according to the region to be recognized by: obtaining a plurality of preset matrix direction parameters; and generating a plurality of target gray level co-occurrence matrices according to the plurality of matrix direction parameters and a preset distance parameter, wherein the gray level co-occurrence matrix comprises the plurality of target gray level co-occurrence matrices, and one matrix direction parameter corresponds to one target gray level co-occurrence matrix.

[0182] As an optional solution, the apparatus is further configured to generate a respective energy parameter, an entropy parameter, a contrast parameter and a correlation parameter according to each target gray level co-occurrence matrix respectively, wherein one target gray level co-occurrence matrix corresponds to one group of target parameters, and the group of target parameters comprises one energy parameter, an entropy parameter, a contrast parameter and a correlation parameter; determine an energy mean value parameter and an energy standard deviation parameter according to the energy parameter in each group of target parameters; determine an entropy mean value parameter and an entropy standard deviation parameter according to the entropy parameter in each group of target parameters; determine a contrast mean value parameter and a contrast standard deviation parameter according to the contrast parameter in each group of target parameters; determine a correlation mean value parameter and a correlation standard deviation parameter according to the correlation parameter in each group of target parameters; and determine the value of the texture feature using the energy mean value parameter and the energy standard deviation parameter, the entropy mean value parameter and the entropy standard deviation parameter, the contrast mean value parameter and the contrast standard deviation parameter, and the correlation mean value parameter and the correlation standard deviation parameter.

[0183] As an optional solution, the apparatus is further configured to generate the gray level co-occurrence matrix according to the region to be recognized by: obtaining a plurality of preset matrix direction parameters; and generating a plurality of target gray level co-occurrence matrices according to the plurality of matrix direction parameters and a preset distance parameter, wherein the gray level co-occurrence matrix comprises the plurality of target gray level co-occurrence matrices, and one matrix direction parameter corresponds to one target gray level co-occurrence matrix.

[0184] As an optional solution, the apparatus is further configured to perform the frequency domain decomposition operation on the region to be recognized to obtain a plurality of subbands and a subband energy corresponding to each subband, and determine the high frequency energy feature based on the subband energy by: obtaining the subband energy of a high frequency subband in the plurality of subbands; and determining the value of the high frequency energy feature as a ratio of the subband energy of the high frequency subband to the subband energy of each subband.

[0185] As an optional solution, the apparatus is configured to perform the thermal sensing feature extraction operation on the to-be-identified region to obtain the thermal sensing feature in the following manner: acquiring temperatures of each pixel point in the to-be-identified region by using the thermal imaging module; determining a temperature distribution feature according to a temperature mean value parameter, a temperature variance parameter and a temperature scale parameter of the pixel points; determining a target pixel point set satisfying a preset temperature condition according to the temperature of each pixel point, and acquiring a mass point pixel point of the to-be-identified region, and determining a centroid feature according to the mass point pixel point and the target pixel point set.

[0186] As an optional solution, the apparatus is configured to determine the centroid feature according to the mass point pixel point and the target pixel point set in the following manner: in a case where the mass point pixel point belongs to the target pixel point set, determining a value of the centroid feature as a positive integer 1; in a case where the mass point pixel point does not belong to the target pixel point set, determining the value of the centroid feature as 0.

[0187] As an optional solution, the apparatus is configured to perform the pyroelectric feature extraction operation on the to-be-identified image to obtain the pyroelectric feature of the to-be-identified region in the following manner: performing the pyroelectric feature extraction operation on the to-be-identified image by using a first waveband filter, a second waveband filter and a third waveband filter to obtain a first pyroelectric feature, a second pyroelectric feature and a third pyroelectric feature, wherein the first waveband filter, the second waveband filter and the third waveband filter are all different; and determining the pyroelectric feature by using the first pyroelectric feature, the second pyroelectric feature and the third pyroelectric feature.

[0188] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0189] Regarding the apparatus in the above embodiments, the specific manner in which each module performs an operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0190] According to an aspect of the present application, a computer program product is provided, which includes a computer program.

[0191] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0192] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section, and / or installed from a removable medium. When the computer program is executed by the central processing unit, various functions defined in the system of the present application are performed.

[0193] In such embodiments, the computer program can be downloaded and installed from a network via the communication section, and / or installed from a removable medium. When the computer program is executed by the central processing unit, various functions provided by embodiments of the present application are performed.

[0194] In other embodiments, the terminal device or server described above can be a node in a distributed system, where the distributed system can be a blockchain system, which can be a distributed system formed by the plurality of nodes connected through network communication. Among them, nodes can form a peer-to-peer network, and any form of computing device, such as a server, a terminal, and an electronic device, can become a node in the blockchain system by joining the peer-to-peer network.

[0195] According to an aspect of the present application, a computer readable storage medium is provided, and a processor of an electronic device reads computer instructions from the computer readable storage medium. The processor executes the computer instructions to cause the electronic device to perform the object recognition method provided in the various optional implementations of the object recognition aspect described above.

[0196] Optionally, in the present embodiment, the computer readable storage medium described above can be configured to store the program for executing the method in the embodiments of the present application.

[0197] Optionally, in the present embodiment, those skilled in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, which can be stored in a computer readable storage medium, and the storage medium can include: flash disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

[0198] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0199] The integrated units in the above embodiments, if implemented in the form of software function units and sold or used as independent products, can be stored in the above computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions to make one or more electronic devices execute all or part of the steps of the methods described in the various embodiments of the present application.

[0200] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0201] In the several embodiments provided in the present application, it should be understood that the disclosed application programs can be implemented in other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0202] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0203] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software function unit.

[0204] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. An object recognition method, characterized in that, include: The region to be identified is determined based on the pixel probability of each pixel in the image to be identified, wherein the pixel probability of each pixel is determined by the first chromaticity component and the second chromaticity component of the pixel, and the pixel probability of the pixels in the region to be identified satisfies the preset probability discrimination condition. Visible light feature extraction and thermal sensing feature extraction operations are performed on the region to be identified to obtain the visible light features and thermal sensing features of the region to be identified. The visible light features include circularity features, texture features and high-frequency energy features, and the thermal sensing features include temperature distribution features and centroid features. A pyroelectric feature extraction operation is then performed on the image to be identified to obtain the pyroelectric features of the region to be identified. The identification result is determined based on the visible light features, the thermal sensing features, and the pyroelectric features, wherein the identification result is used to indicate whether the image to be identified contains a target object.

2. The method according to claim 1, characterized in that, The step of determining the identification result based on the visible light characteristics, the thermal sensing characteristics, and the pyroelectric characteristics includes: By fusing the visible light features, the thermal sensing features, and the pyroelectric features, a target feature vector is obtained; The recognition result is determined based on the target feature vector, wherein the recognition result is used to indicate whether the image to be recognized contains a flame, and the target object includes a flame.

3. The method according to claim 1, characterized in that, The visible light feature extraction operation and the thermal sensing feature extraction operation are performed on the region to be identified to obtain the visible light feature and thermal sensing feature of the region to be identified, including at least one of the following: Obtain the perimeter and area parameters of the region to be identified; determine the circularity feature based on the product of the area parameters and the perimeter parameters; A gray-level co-occurrence matrix is ​​generated based on the region to be identified, and energy parameters, entropy parameters, contrast parameters, and correlation parameters are generated based on the gray-level co-occurrence matrix; the texture features are determined based on the energy parameters, entropy parameters, contrast parameters, and correlation parameters. A frequency domain decomposition operation is performed on the region to be identified to obtain multiple sub-bands and the sub-band energy corresponding to each sub-band. The high-frequency energy characteristics are determined based on the sub-band energy.

4. The method according to claim 3, characterized in that, Determining the roundness feature based on the product parameter of the area parameters and the perimeter parameter includes: The product of the preset parameter and the product parameter, and the square of the perimeter parameter are used to determine the value of the roundness feature, wherein the preset parameter is a positive integer greater than 0.

5. The method according to claim 3, characterized in that, The step of generating a gray-level co-occurrence matrix based on the region to be identified includes: Obtain multiple preset matrix direction parameters; Multiple target gray-level co-occurrence matrices are generated according to multiple matrix direction parameters and preset distance parameters, wherein the gray-level co-occurrence matrix includes multiple target gray-level co-occurrence matrices, and one matrix direction parameter corresponds to one target gray-level co-occurrence matrix.

6. The method according to claim 5, characterized in that, The method further includes: Each target gray-level co-occurrence matrix generates its corresponding energy parameter, entropy parameter, contrast parameter, and correlation parameter. Each target gray-level co-occurrence matrix corresponds to a set of target parameters, which includes an energy parameter, an entropy parameter, a contrast parameter, and a correlation parameter. The energy mean parameter and energy standard deviation parameter are determined based on the energy parameter in each set of target parameters; the entropy mean parameter and entropy standard deviation parameter are determined based on the entropy parameter in each set of target parameters; the contrast mean parameter and contrast standard deviation parameter are determined based on the contrast parameter in each set of target parameters; the correlation mean parameter and correlation standard deviation parameter are determined based on the correlation parameter in each set of target parameters. The values ​​of the texture features are determined using the mean energy parameter and the standard deviation energy parameter, the mean entropy parameter and the standard deviation entropy parameter, the mean contrast parameter and the standard deviation contrast parameter, and the mean correlation parameter and the standard deviation correlation parameter.

7. The method according to claim 3, characterized in that, The step of generating a gray-level co-occurrence matrix based on the region to be identified, and generating energy parameters, entropy parameters, contrast parameters, and correlation parameters based on the gray-level co-occurrence matrix, includes: The energy parameter is determined by summing the squares of each element in the gray-level co-occurrence matrix. The entropy parameter is determined by summing the negative value of the product of each element in the gray-level co-occurrence matrix and the logarithm of that element. The contrast parameter is determined by the sum of the squares of the differences between the value of each element in the gray-level co-occurrence matrix and the row and column indices of the corresponding element. The correlation parameter is determined by subtracting the product of the row index and column index of each element in the gray-level co-occurrence matrix from the sum of the products of the row index and column index of the corresponding element position, and then dividing by the product of the row index standard deviation and column index standard deviation.

8. The method according to claim 3, characterized in that, The step of performing frequency domain decomposition on the region to be identified to obtain multiple sub-bands and the sub-band energy corresponding to each sub-band, and determining the high-frequency energy characteristics based on the sub-band energy, includes: Obtain the subband energy of the high-frequency subband among the multiple subbands; The ratio of the subband energy of the high-frequency subband to the subband energy of each subband is determined as the value of the high-frequency energy characteristic.

9. The method according to claim 1, characterized in that, The step of performing thermal sensing feature extraction on the region to be identified to obtain thermal sensing features includes: The temperature of each pixel in the region to be identified is obtained using a thermal imaging module; the temperature distribution characteristics are determined based on the mean temperature parameter, temperature variance parameter, and temperature scale parameter of the pixel. Based on the temperature of each pixel, a set of target pixels that meet the preset temperature conditions is determined, and the mass pixels of the region to be identified are obtained. The centroid feature is determined based on the mass pixels and the set of target pixels.

10. The method according to claim 9, characterized in that, Determining the centroid feature based on the set of mass pixels and the set of target pixels includes: When the mass point pixel belongs to the target pixel set, the value of the centroid feature is determined to be a positive integer 1; If the centroid pixel does not belong to the target pixel set, the value of the centroid feature is set to 0.

11. The method according to claim 1, characterized in that, The step of performing pyroelectric feature extraction on the image to be identified to obtain the pyroelectric features of the region to be identified includes: The image to be identified is subjected to pyroelectric feature extraction using a first-band filter, a second-band filter, and a third-band filter to obtain a first pyroelectric feature, a second pyroelectric feature, and a third pyroelectric feature, wherein the first-band filter, the second-band filter, and the third-band filter are all different. The pyroelectric feature is determined using the first pyroelectric feature, the second pyroelectric feature, and the third pyroelectric feature.

12. An object recognition device, characterized in that, include: The determination module is used to determine the region to be identified based on the pixel probability of each pixel in the image to be identified, wherein the pixel probability of each pixel is determined by the first chromaticity component and the second chromaticity component of the pixel, and the pixel probability of the pixels in the region to be identified satisfies a preset probability discrimination condition. The execution module is used to perform visible light feature extraction and thermal sensing feature extraction operations on the region to be identified, respectively, to obtain the visible light features and thermal sensing features of the region to be identified. The visible light features include circularity features, texture features, and high-frequency energy features, and the thermal sensing features include temperature distribution features and centroid features. The module also performs pyroelectric feature extraction on the image to be identified to obtain the pyroelectric features of the region to be identified. The identification module is used to determine the identification result based on the visible light feature, the thermal sensing feature and the pyroelectric feature, wherein the identification result is used to indicate whether the image to be identified contains a target object.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 11.

14. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1 to 11.

15. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 11 through the computer program.

Images