Radio frequency detection positioning method and system combined with multi-mode vision technology
By combining multimodal vision technology and radio frequency detection technology, the problem of difficult to identify and locate hidden abnormal electronic devices in complex environments is solved, efficient detection and precise positioning are achieved, and the level of safety protection is improved.
Patent Information
- Application Number
- CN202411769085.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to effectively identify and locate hidden abnormal electronic devices in complex environments, resulting in difficulty in improving the accuracy and reliability of security detection.
Combining multimodal vision technology and radio frequency detection technology, data acquisition is carried out through infrared detection, radio frequency scanning and low light night vision, heat source point clustering, early warning position mapping and abnormal analysis are carried out to achieve efficient detection of hidden equipment in complex environments.
It significantly improves the identification and positioning accuracy of abnormal electronic equipment, realizes efficient detection of hidden equipment in complex environments, and improves the overall safety protection level.
Smart Images
Figure CN119540655A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field related to intelligent detection technology, and specifically to a radio frequency detection and positioning method and system combined with multimodal vision technology. Background Art
[0002] In the current field of information security, preventing and detecting hidden abnormal electronic devices, such as hidden cameras and hidden recording devices, has become an important research direction. With the development of hidden device technology, these devices are becoming more and more miniaturized and intelligent, and can be monitored without being noticed. Traditional detection methods, such as manual inspection or simple RF signal detection, are often difficult to effectively identify these devices in complex environments.
[0003] Therefore, in the prior art, traditional detection methods are difficult to effectively identify and locate hidden abnormal electronic devices in complex environments, resulting in a technical problem that the accuracy and reliability of security detection are difficult to improve. Summary of the invention
[0004] This application solves the technical problem that the traditional detection means in the prior art are difficult to effectively identify and locate hidden abnormal electronic devices in complex environments, resulting in the difficulty in improving the accuracy and reliability of security detection by providing a radio frequency detection and positioning method and system combined with multimodal vision technology. By combining multimodal vision technology and radio frequency detection technology, the recognition and positioning accuracy of abnormal electronic devices can be significantly improved, and efficient detection of hidden devices in complex environments can be achieved, thereby improving the overall security protection level.
[0005] The present application provides a radio frequency detection and positioning method combined with multimodal vision technology, the method is applied to a detection device, the detection device includes a radio frequency detection unit, a low-light night vision unit and an infrared detection unit, the method includes: based on the infrared detection unit, an image of the detection requirement area is collected to obtain first image data. The first image data is preprocessed, and multiple heat source points are clustered based on the preprocessing result to obtain a heat source area clustering result. The detection requirement area is scanned based on the radio frequency detection unit to obtain a scan warning position. The scan warning position is mapped to the preprocessing result to obtain a mapping image. The mapping image is used in combination with the heat source area clustering result to perform an abnormal combination analysis, obtain the heat source target attribute and the abnormal target position, and obtain a mapping image analysis result. The real-time warning analysis position in the mapping image analysis result is extracted, and the low-light night vision unit is used to collect low-light images to obtain a second image set. Abnormal device determination is performed on the second image set to obtain abnormal devices and locate the abnormal device position.
[0006] In the implementation, the first image data is preprocessed, and multiple heat source points are clustered based on the preprocessing result to obtain the heat source area clustering result, including: preprocessing the first image data to obtain the preprocessing result, and the preprocessing includes denoising and grayscale conversion. Multiple heat source points are clustered on the preprocessing result to obtain the heat source clustering area. Based on the heat source clustering area, the same area is merged according to the preset normalization parameter to obtain the heat source area clustering result.
[0007] In the implementation, the preprocessing result is clustered with multiple heat source points to obtain the heat source clustering area, including: traversing the grayscale image of the preprocessing result to obtain multiple grayscale extreme value points. The multiple grayscale extreme value points are used as the initial clustering center, and the grayscale values of the adjacent pixels of the initial clustering center are traversed according to the preset gradient parameters to obtain the pixels that meet the preset gradient parameters as the initial clustering area of the initial clustering center. The initial clustering area is iteratively diffused based on the preset gradient parameters until the grayscale values of the adjacent pixels of all pixels cannot meet the preset gradient parameters, thereby obtaining the heat source clustering area.
[0008] In an implementation, scanning the detection requirement area based on the radio frequency detection unit to obtain a scanning warning position includes: continuously scanning the detection requirement area using the radio frequency detection unit. Collecting multiple starting positions and multiple ending positions of the radio frequency detection unit warning signal in the continuous scanning path. Based on the multiple starting positions and the multiple ending positions, the scanning warning position is obtained.
[0009] In an implementation, mapping the scanned warning position to the preprocessing result to obtain a mapping image includes: based on the correspondence between pixels and actual distances, converting the scanned warning position into warning position pixel coordinates. Pixel mapping of the preprocessing result is performed according to the warning position pixel coordinates to obtain a mapping image.
[0010] In the implementation, obtaining the mapping image analysis result includes: performing pixel overlap on the mapping image and the heat source area clustering result, obtaining the positional relationship between the scanning warning position and the multi-heat source point area, wherein the positional relationship includes adjacent, intersecting and containing. Based on the positional relationship, the pixel group where the scanning warning position and the multi-heat source point area overlap is obtained to obtain the area overlap ratio. Based on the area overlap ratio, the positional relationship is screened for associated areas to obtain associated positions and non-associated positions. All associated areas in the associated position are marked with abnormal heat source target attributes, and the heat source extreme value points of each associated area are extracted as abnormal target positions. All scanning warning positions of the non-associated position are marked with abnormal target attributes, and the scanning warning position is used as the abnormal target position, and all multi-heat source point areas of the non-associated position are marked with normal heat source target attributes. Based on the abnormal heat source target attribute mark, the abnormal target attribute mark and the corresponding abnormal target position, a real-time warning analysis position is obtained, and based on the normal heat source target attribute mark, a non-warning position is obtained. Based on the real-time warning analysis position and the non-warning position, the mapping image analysis result is obtained.
[0011] In an implementation, performing abnormal device determination on the second image set, obtaining abnormal devices and locating abnormal device positions includes: performing image segmentation on the second image set according to the real-time warning analysis position, obtaining image segmentation results. Inputting the image segmentation results into an abnormal device determination model, performing image abnormality determination, obtaining abnormal values of the image segmentation. Determining abnormal devices based on the abnormal values of the image segmentation, and obtaining abnormal device positions.
[0012] The present application also provides a radio frequency detection and positioning system combined with multimodal vision technology, wherein the system is communicatively connected with a radio frequency detection unit, a low light night vision unit and an infrared detection unit, and the system comprises: The infrared image acquisition module is used to collect the image of the detection required area based on the infrared detection unit to obtain the first image data.
[0013] The heat source clustering module is used to preprocess the first image data, cluster multiple heat source points based on the preprocessing results, and obtain heat source area clustering results.
[0014] The scanning warning position acquisition module is used to scan the detection requirement area based on the radio frequency detection unit to obtain the scanning warning position.
[0015] A mapping image acquisition module is used to map the scan warning position to the preprocessing result to acquire a mapping image.
[0016] The image analysis module is used to use the mapping image in combination with the heat source area clustering result to perform anomaly combination analysis, obtain the heat source target attributes and the abnormal target position, and obtain the mapping image analysis result.
[0017] The low-light-level image acquisition module is used to extract the real-time warning analysis position in the mapping image analysis result, and use the low-light-level night vision unit to collect low-light images to obtain a second image set.
[0018] The abnormal device location acquisition module is used to perform abnormal device determination on the second image set, acquire abnormal devices and locate abnormal device locations.
[0019] The radio frequency detection and positioning method and system proposed in this application in combination with multimodal vision technology collects the detection demand area image based on the infrared detection unit to obtain the first image data. The first image data is preprocessed, and multiple heat source points are clustered based on the preprocessing results to obtain the heat source area clustering results. The detection demand area is scanned based on the radio frequency detection unit to obtain the scanning warning position. The scanning warning position is mapped to the preprocessing result to obtain the mapping image. The mapping image is used in combination with the heat source area clustering result to perform abnormal combination analysis, obtain the heat source target attribute and the abnormal target position, and obtain the mapping image analysis result. The real-time warning analysis position in the mapping image analysis result is extracted, and the low-light night vision unit is used to collect low-light images to obtain the second image set. Abnormal device determination is performed on the second image set to obtain the abnormal device and locate the abnormal device position. The technical problem that the traditional detection means in the prior art are difficult to effectively identify and locate hidden abnormal electronic devices in a complex environment, resulting in the difficulty in improving the accuracy and reliability of security detection is solved. By combining multimodal vision technology and radio frequency detection technology, the accuracy of identifying and locating abnormal electronic equipment can be significantly improved, and efficient detection of hidden equipment in complex environments can be achieved, thereby improving the overall security protection level. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.
[0021] Figure 1 A schematic diagram of the flow of a radio frequency detection and positioning method combined with multimodal vision technology provided in an embodiment of the present application; Figure 2A schematic diagram of the structure of a radio frequency detection and positioning system combined with multimodal vision technology provided in an embodiment of the present application.
[0022] Explanation of the reference numerals: infrared image acquisition module 11 , heat source clustering module 12 , scanning warning position acquisition module 13 , mapping image acquisition module 14 , image analysis module 15 , low-light image acquisition module 16 , abnormal equipment position acquisition module 17 . DETAILED DESCRIPTION
[0023] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.
[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0025] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.
[0026] The present application embodiment provides a radio frequency detection and positioning method and system combined with multimodal vision technology, the method is applied to a detection device, the detection device includes a radio frequency detection unit, a low light night vision unit and an infrared detection unit, such as Figure 1 As shown, the method includes: Based on the infrared detection unit, an image of the detection required area is collected to obtain first image data. The first image data is preprocessed, and multiple heat source points are clustered based on the preprocessing result to obtain a heat source area clustering result. Based on the radio frequency detection unit, the detection required area is scanned to obtain a scanning warning position. The scanning warning position is mapped to the preprocessing result to obtain a mapping image.
[0027] The detection device includes a radio frequency detection unit, a low-light night vision unit and an infrared detection unit. The radio frequency detection unit is a tool for detecting electronic devices by emitting high-frequency radio frequency signals. The low-light night vision unit is used for observation and monitoring under extremely low light conditions. The infrared detection unit is used to collect images of heat or infrared radiation. The infrared detection unit collects the image of the detection requirement area to obtain the first image data, and the first image data needs to completely cover the detection requirement area. Subsequently, the first image data is preprocessed, and the preprocessing steps include denoising and grayscale conversion. Based on the preprocessing results, multiple heat source points are clustered to obtain the heat source area clustering results. The radio frequency detection unit is used to scan the detection requirement area, detect the source of the radio frequency signal, obtain the starting position and the ending position of multiple radio frequency signals, and thus determine the scanning warning position. Further, by aligning the warning position of the radio frequency signal, that is, the coordinate position of the actual environment, with the data of the infrared image, a mapping image is obtained.
[0028] The method provided in the embodiment of the present application further includes: preprocessing the first image data to obtain the preprocessing result, wherein the preprocessing includes denoising and grayscale conversion. The preprocessing result is clustered into multiple heat source points to obtain a heat source clustering area. Based on the heat source clustering area, the same area is merged according to a preset normalization parameter to obtain a heat source area clustering result.
[0029] The first image data is preprocessed, and multiple heat source points are clustered based on the preprocessing results to obtain heat source area clustering results, including: using Gaussian filtering to denoise the infrared grayscale image of the first image data, smoothing the high-frequency noise in the image while retaining important edge information, and then performing grayscale conversion on the denoised image to obtain the preprocessing result, wherein the preprocessing includes denoising and grayscale conversion. Subsequently, multiple heat source points are clustered on the preprocessing result to obtain the heat source clustering area. During the clustering process, multiple closely adjacent or partially overlapping clustering areas may be generated. In order to improve the accuracy of heat source identification, these areas need to be merged. Based on the heat source clustering area, the same area is merged according to the preset normalization parameter, and the preset normalization parameter is a preset pixel distance parameter. When the distance between two areas is less than the preset normalization parameter, the heat sources of the corresponding areas are generated by the same device, and the two areas are merged. When the distance between the two areas is greater than the preset normalization parameter, the heat sources of the corresponding areas are generated by different devices, and the two areas are not merged to ensure the uniqueness of the correspondence between the area and the running device, so as to reduce the subsequent regional processing volume of other devices and obtain the heat source area clustering result.
[0030] The method provided by the embodiment of the present application also includes: traversing the grayscale image of the preprocessing result to obtain multiple grayscale extreme value points. Using the multiple grayscale extreme value points as the initial cluster center, traversing the grayscale values of the adjacent pixels of the initial cluster center according to the preset gradient parameters, and obtaining the pixels that meet the preset gradient parameters as the initial cluster area of the initial cluster center. Based on the preset gradient parameters, iterative diffusion of the initial cluster area is performed until the grayscale values of the adjacent pixels of all pixels cannot meet the preset gradient parameters, and the heat source cluster area is obtained.
[0031] Traverse the grayscale image of the preprocessing result, and extract multiple grayscale extreme value points in the grayscale image pixels. Grayscale extreme value points refer to pixels in the image whose grayscale values are greater than the grayscale median value of the image. When the abnormal equipment is running, a heat source significantly higher than the environment will be generated, which usually corresponds to the hottest area. By extracting these points, the main heat source position in the image can be determined. Subsequently, the multiple grayscale extreme value points are used as initial cluster centers, wherein the number of initial cluster centers is the same as the number of grayscale extreme value points. Traverse the grayscale values of the adjacent pixels of the initial cluster center according to the preset gradient parameter, the preset gradient parameter is the maximum value of the gradient of the grayscale value, when the pixel value gradient does not meet the gradient value, i.e., it is greater than the gradient value, the gradient changes significantly, and the edge area of the heat source may be traversed, and the pixels that meet the preset gradient parameter, i.e., less than or equal to the preset gradient parameter, are obtained as the initial cluster area of the initial cluster center. That is, starting from each grayscale extreme value point, diffuse to the surrounding according to the preset gradient parameter, and traverse the grayscale values of the adjacent pixels one by one. If the grayscale value of the adjacent pixel is within the specified gradient range, it is included in the initial cluster area. For example, if the gradient parameter is set to ±20 and the gray value of the initial center is 200, then all adjacent pixels with gray values between 180 and 220 will be added to the initial clustering region. The initial clustering region is iteratively diffused based on the preset gradient parameter. Iterative diffusion refers to expanding outward from the initial clustering region by continuously checking the gray values of adjacent pixels until there are no more pixels that meet the preset gradient parameters. The diffusion process stops and eventually forms a complete heat source clustering region to obtain the heat source clustering region. Based on the above process, it can be ensured that all relevant heat source pixels are included in the clustering region.
[0032] The method provided in the embodiment of the present application further includes: using the radio frequency detection unit to continuously scan the detection requirement area, collecting multiple starting positions and multiple ending positions of the radio frequency detection unit warning signal in the continuous scanning path, and obtaining the scanning warning position based on the multiple starting positions and the multiple ending positions.
[0033] The radio frequency detection unit is used to move the detection required area at a fixed speed and serpentine path, and the entire detection area is scanned uninterruptedly step by step to ensure that all possible signal sources are covered. Collect multiple starting positions and multiple ending positions of the warning signal of the radio frequency detection unit in the continuous scanning path. When the radio frequency detection unit detects an abnormal radio frequency signal, such as the signal strength exceeds the preset threshold, the device sends a warning signal, indicating that there is a potential radio frequency signal source such as a hidden device at this position. When leaving the potential radio frequency signal source, the warning signal will disappear, and the last position recorded at this time where the warning signal exists is the ending position. The position where the radio frequency detection unit first detects the warning signal after the ending position or the position where the warning signal is first detected is the starting position, wherein each starting position corresponds to the ending position one by one, and the multiple starting positions and the multiple ending positions are all coordinate positions in the actual environment, and the coordinate origin is the lower left corner of the area to be detected, which can be set according to the actual situation. Based on the multiple starting positions and the multiple ending positions, traverse each starting position and ending position to see if there are adjacent starting positions and ending positions in the vertical direction. If so, connect the adjacent starting positions and ending positions by area to form a warning position. If not, take the area of the current starting position and ending position as a warning position, traverse all the multiple starting positions and the multiple ending positions, and obtain multiple warning positions to obtain the scanned warning position, which includes an area composed of multiple warning positions.
[0034] The method provided in the embodiment of the present application further includes: based on the correspondence between pixels and actual distances, converting the scanned warning position into warning position pixel coordinates, and performing pixel mapping of the preprocessing result according to the warning position pixel coordinates to obtain a mapping image.
[0035] Based on the correspondence between pixels and actual distances, the actual distance is converted into pixel distance, and the scanned warning position is converted into the warning position pixel coordinates. Pixel mapping of the preprocessing result is performed according to the warning position pixel coordinates to obtain a mapping image, in which the pixel area formed by the scanned warning position is included.
[0036] The mapping image is combined with the heat source area clustering result to perform an abnormal combination analysis, obtain the heat source target attributes and the abnormal target position, and obtain the mapping image analysis result. The real-time warning analysis position in the mapping image analysis result is extracted, and the low-light night vision unit is used to collect low-light images to obtain a second image set. The second image set is used to perform abnormal device determination, obtain abnormal devices and locate the abnormal device positions.
[0037] The mapping image is combined with the heat source area clustering result to perform an abnormal combination analysis, obtain the heat source target attributes and the abnormal target position, and obtain the mapping image analysis result, which includes the real-time warning analysis position and the non-warning position. Further, the real-time warning analysis position in the mapping image analysis result is extracted, and the low-light night vision unit is used to collect low-light images at the real-time warning analysis position coordinates, so as to obtain a second image set of the corresponding position. The abnormal device determination algorithm is used to perform abnormal device determination on the second image set, confirm and locate the position of abnormal devices such as cameras, and take corresponding safety measures. It solves the technical problem that traditional detection means in the prior art are difficult to effectively identify and locate hidden abnormal electronic devices in complex environments, resulting in the difficulty in improving the accuracy and reliability of security detection. By combining multimodal vision technology and radio frequency detection technology, the recognition and positioning accuracy of abnormal electronic devices can be significantly improved, and efficient detection of hidden devices in complex environments can be achieved, thereby improving the overall safety protection level.
[0038] The method provided in the embodiment of the present application also includes: overlapping pixels of the mapping image and the clustering result of the heat source area to obtain the positional relationship between the scanning warning position and the multi-heat source point area, wherein the positional relationship includes adjacent, intersecting and containing. Based on the positional relationship, the pixel group where the scanning warning position and the multi-heat source point area overlap is obtained to obtain the area overlap ratio. Based on the area overlap ratio, the positional relationship is screened for associated areas to obtain associated positions and non-associated positions. All associated areas in the associated position are marked as abnormal for the heat source target attribute, and the heat source extreme value points of each associated area are extracted as abnormal target positions. All scanning warning positions of the non-associated position are marked as abnormal for the target attribute, and the scanning warning position is used as the abnormal target position, and all multi-heat source point areas of the non-associated position are marked as normal for the heat source target attribute. Based on the abnormal mark of the heat source target attribute, the abnormal mark of the target attribute and the corresponding abnormal target position, a real-time warning analysis position is obtained, and based on the normal mark of the heat source target attribute, a non-warning position is obtained. Based on the real-time warning analysis position and the non-warning position, a mapping image analysis result is obtained.
[0039] The mapping image and the heat source clustering result are registered so that the mapping image and the heat source clustering result are analyzed in the same coordinate system. Subsequently, the mapping image and the heat source clustering result are subjected to pixel overlap analysis to identify the specific positional relationship between the RF warning position and the multi-heat source point area. For example, a certain RF warning position in the mapping image overlaps with the heat source clustering area and is identified as an intersection relationship. If the RF warning position is completely contained in the heat source area, it is marked as an inclusion relationship. If the RF warning position is adjacent to the heat source area, it is an adjacent relationship. Subsequently, based on the positional relationship, the pixel group where the scanning warning position overlaps with the multi-heat source point area is obtained, and the ratio of the overlapping pixel group to the scanning warning position pixel is calculated to obtain the regional overlap ratio. Exemplarily, the warning position has 100 pixels, 60 of which overlap with the heat source area pixels, and the overlap ratio is 60%. The overlap ratio can be used to judge the strength of the association between the two, thereby providing a basis for subsequent regional screening. Further, an overlap ratio threshold (for example, 50%) is set, and all warning positions are divided into associated positions and non-associated positions according to this threshold. For areas where the overlap ratio exceeds the threshold, they are marked as associated locations, and for areas where the overlap ratio is below the threshold, they are marked as non-associated locations.
[0040] Furthermore, all associated areas of the associated position are marked as abnormal in the heat source target attribute, the heat source extreme value points are extracted and the pixel coordinates are converted into actual coordinates, and the actual coordinate position after the conversion is used as the abnormal target position. The heat source extreme value point refers to one or more points with the highest temperature in the heat source area, which are usually used as the most likely position of the abnormal target. All scanning warning positions in the non-associated position are marked as abnormal in the target attribute, and these positions are converted into pixel coordinates and actual coordinates, and the actual coordinate position after the conversion is directly marked as the abnormal target position. For the heat source area of the non-associated position, it is marked as normal in the heat source target attribute, indicating that these areas are not related to the radio frequency signal and are usually normal heat sources. The analysis results of all associated areas and non-associated areas are summarized, and the real-time warning analysis position is obtained based on the heat source target attribute abnormality mark, the target attribute abnormality mark and the corresponding abnormal target position, and the area marked as abnormal is extracted as the real-time warning analysis position for further analysis. The normally marked area is regarded as a non-warning position, archived but not as a key focus object. Based on the real-time warning analysis position and the non-warning position, a mapping image analysis result is obtained, in which the real-time warning analysis position and the non-warning position are clearly marked, and the real-time warning analysis position and the non-warning position are both actual coordinate positions, which are used for subsequent analysis and processing of the low-light night vision unit to identify abnormal devices.
[0041] The method provided in the embodiment of the present application further includes: performing image segmentation on the second image set according to the real-time warning analysis position, and obtaining image segmentation results. The image segmentation results are input into the abnormal device determination model, image abnormality determination is performed, and abnormal values of the image segmentation are obtained. The abnormal device is determined based on the abnormal value of the image segmentation, and the abnormal device position is obtained.
[0042] The second image set is divided into images according to the real-time warning analysis position, and the image division result is obtained, and the image division result includes each real-time warning analysis position and the corresponding image in the second image set. Constructing an abnormal device determination model includes collecting a large number of images of abnormal devices collected by a low-light night vision unit in different directions under different lighting environments, such as hidden cameras, recording devices, reflective points, etc. The collected images are annotated to mark the abnormal device positions in the images. The data set is expanded by data enhancement techniques such as rotation, cropping, adjusting brightness, and adding noise. The normal device data is obtained in the same way to expand the extended data set. Based on the extended data set and the corresponding annotated abnormal device positions as training data, training is performed based on the YOLO model until the output result of the model meets the preset accuracy rate to obtain the abnormal device determination model. The image division result is input into the abnormal device determination model, and the image abnormality determination is performed to obtain the target category, the center coordinate, and the confidence score. The target category includes the normal device category and the abnormal device category, and the abnormal value of the image division is obtained. The abnormal value is the confidence output when the model outputs the abnormal device category. The abnormal device is determined based on the abnormal value of the image division. When the abnormal value of the image division is greater than the set threshold, the corresponding abnormal device identification category can be directly determined, and the abnormal device position can be obtained based on the corresponding relationship between the output center coordinates and the actual coordinates.
[0043] In the above, refer to Figure 1 The radio frequency detection and positioning method combined with multimodal vision technology according to an embodiment of the present invention is described in detail. Figure 2 A radio frequency detection and positioning system combined with multimodal vision technology according to an embodiment of the present invention is described.
[0044] The radio frequency detection and positioning system combined with multimodal vision technology according to the embodiment of the present invention solves the technical problem that the traditional detection means in the prior art are difficult to effectively identify and locate hidden abnormal electronic devices in complex environments, resulting in the difficulty in improving the accuracy and reliability of security detection. By combining multimodal vision technology and radio frequency detection technology, the recognition and positioning accuracy of abnormal electronic devices can be significantly improved, and efficient detection of hidden devices in complex environments can be achieved, thereby improving the overall security protection level. The radio frequency detection and positioning system combined with multimodal vision technology, the system is communicatively connected with the radio frequency detection unit, the low-light night vision unit and the infrared detection unit, and includes: an infrared image acquisition module 11, a heat source clustering module 12, a scanning warning position acquisition module 13, a mapping image acquisition module 14, an image analysis module 15, a low-light image acquisition module 16, and an abnormal device position acquisition module 17.
[0045] The infrared image acquisition module 11 is used to collect the image of the detection required area based on the infrared detection unit to obtain the first image data.
[0046] The heat source clustering module 12 is used to preprocess the first image data, perform multi-heat source point clustering based on the preprocessing results, and obtain heat source area clustering results.
[0047] The scanning warning position acquisition module 13 is used to scan the detection requirement area based on the radio frequency detection unit to acquire the scanning warning position.
[0048] The mapping image acquisition module 14 is used to map the scan warning position to the preprocessing result to acquire a mapping image.
[0049] The image analysis module 15 is used to use the mapping image in combination with the heat source area clustering result to perform anomaly combination analysis, obtain heat source target attributes and abnormal target locations, and obtain mapping image analysis results.
[0050] The low-light image acquisition module 16 is used to extract the real-time warning analysis position in the mapping image analysis result, and use the low-light night vision unit to collect low-light images to obtain a second image set.
[0051] The abnormal device location acquisition module 17 is used to perform abnormal device determination on the second image set, acquire abnormal devices and locate the abnormal device locations.
[0052] The specific configuration of the heat source clustering module 12 will be described in detail below. The heat source clustering module 12 may further include: preprocessing the first image data, clustering multiple heat source points based on the preprocessing result, and obtaining the heat source area clustering result, including: preprocessing the first image data to obtain the preprocessing result, wherein the preprocessing includes denoising and grayscale conversion. Clustering multiple heat source points on the preprocessing result to obtain a heat source clustering area. Based on the heat source clustering area, the same area is merged according to the preset normalization parameter to obtain the heat source area clustering result.
[0053] The specific configuration of the heat source clustering module 12 will be described in detail below. The heat source clustering module 12 further includes: clustering multiple heat source points on the preprocessing results to obtain a heat source clustering area, including: traversing the grayscale image of the preprocessing results to obtain multiple grayscale extreme value points. The multiple grayscale extreme value points are used as the initial clustering center, and the grayscale values of the adjacent pixels of the initial clustering center are traversed according to the preset gradient parameters to obtain pixels that meet the preset gradient parameters as the initial clustering area of the initial clustering center. The initial clustering area is iteratively diffused based on the preset gradient parameters until the grayscale values of the adjacent pixels of all pixels cannot meet the preset gradient parameters to obtain the heat source clustering area.
[0054] The specific configuration of the scanning warning position acquisition module 13 will be described in detail below. The scanning warning position acquisition module 13 may further include: scanning the detection requirement area based on the radio frequency detection unit to obtain the scanning warning position, including: continuously scanning the detection requirement area using the radio frequency detection unit. Collect multiple starting positions and multiple ending positions of the warning signal of the radio frequency detection unit in the continuous scanning path. Based on the multiple starting positions and the multiple ending positions, the scanning warning position is obtained.
[0055] The specific configuration of the mapping image acquisition module 14 will be described in detail below. The mapping image acquisition module 14 further includes: mapping the scanned warning position to the preprocessing result to obtain a mapping image, including: based on the correspondence between the pixel and the actual distance, converting the scanned warning position into the warning position pixel coordinates. Performing pixel mapping of the preprocessing result according to the warning position pixel coordinates to obtain a mapping image.
[0056] The specific configuration of the image analysis module 15 will be described in detail below. The image analysis module 15 further includes: obtaining the mapping image analysis result, including: pixel overlapping of the mapping image and the heat source area clustering result, obtaining the positional relationship between the scanning warning position and the multi-heat source point area, wherein the positional relationship includes adjacent, intersecting and containing. Based on the positional relationship, the pixel group where the scanning warning position and the multi-heat source point area overlap is obtained to obtain the area overlap ratio. Based on the area overlap ratio, the positional relationship is screened for associated areas to obtain associated positions and non-associated positions. All associated areas in the associated position are marked with abnormal heat source target attributes, and the heat source extreme value points of each associated area are extracted as abnormal target positions. All scanning warning positions of the non-associated position are marked with abnormal target attributes, and the scanning warning position is used as the abnormal target position, and all multi-heat source point areas of the non-associated position are marked with normal heat source target attributes. Based on the abnormal heat source target attribute mark, the target attribute abnormal mark and the corresponding abnormal target position, a real-time warning analysis position is obtained, and based on the normal heat source target attribute mark, a non-warning position is obtained. Based on the real-time warning analysis position and the non-warning position, a mapping image analysis result is obtained.
[0057] The specific configuration of the abnormal device location acquisition module 17 will be described in detail below. The abnormal device location acquisition module 17 further includes: performing abnormal device determination on the second image set, acquiring abnormal devices and locating abnormal device locations, including: performing image segmentation on the second image set according to the real-time warning analysis location, and acquiring image segmentation results. The image segmentation results are input into the abnormal device determination model, image abnormality determination is performed, and abnormal values of image segmentation are acquired. The abnormal devices are determined based on the abnormal values of the image segmentation, and the abnormal device locations are obtained.
[0058] The radio frequency detection and positioning system combined with multimodal vision technology provided in the embodiment of the present invention can execute the radio frequency detection and positioning method combined with multimodal vision technology provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0059] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or the server, and the various units and modules included are only divided according to functional logic, but are not limited to the above divisions, as long as the corresponding functions can be achieved. In addition, the specific names of the functional units are only for the convenience of distinguishing from each other, and are not used to limit the scope of protection of the present invention.
[0060] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A radio frequency detection and positioning method combining multimodal vision technology, characterized in that: The method is applied to a detection device, the detection device includes a radio frequency detection unit, a low light night vision unit and an infrared detection unit, and the method includes: Based on the infrared detection unit collecting the detection required area image, obtaining first image data; Preprocessing the first image data, performing multi-heat source point clustering based on the preprocessing result, and obtaining a heat source area clustering result; Scanning the detection requirement area based on the radio frequency detection unit to obtain a scanning warning position; Mapping the scan warning position to the preprocessing result to obtain a mapping image; Using the mapping image in combination with the heat source area clustering result to perform anomaly combination analysis, obtain heat source target attributes and abnormal target locations, and obtain mapping image analysis results; Extracting the real-time warning analysis position in the mapping image analysis result, and using the low-light-level night vision unit to collect low-light images to obtain a second image set; An abnormal device determination is performed on the second image set to obtain the abnormal device and locate the position of the abnormal device.
2. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1 is characterized in that: Preprocessing the first image data, clustering multiple heat source points based on the preprocessing results, and obtaining heat source area clustering results, including: Preprocessing the first image data to obtain the preprocessing result, wherein the preprocessing includes denoising and grayscale conversion; Performing multi-heat source point clustering on the preprocessing results to obtain heat source clustering areas; Based on the heat source clustering regions, the same regions are merged according to preset normalization parameters to obtain heat source region clustering results.
3. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1 is characterized in that: The preprocessing results are clustered into multiple heat source points to obtain heat source clustering areas, including: Traversing the grayscale image of the preprocessing result to obtain a plurality of grayscale extreme value points; Taking the multiple grayscale extreme value points as initial cluster centers, traversing the grayscale values of adjacent pixels of the initial cluster centers according to preset gradient parameters, and obtaining pixels satisfying the preset gradient parameters as initial cluster regions of the initial cluster centers; The initial clustering region is iteratively diffused based on the preset gradient parameter until the grayscale values of adjacent pixels of all pixels cannot satisfy the preset gradient parameter, thereby obtaining the heat source clustering region.
4. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1, characterized in that: Scanning the detection requirement area based on the radio frequency detection unit to obtain a scanning warning position includes: Utilizing the radio frequency detection unit to continuously scan the detection required area; Collecting multiple starting positions and multiple ending positions of the warning signal of the radio frequency detection unit in the continuous scanning path; The scanning warning position is acquired based on the multiple starting positions and the multiple ending positions.
5. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1, characterized in that: Mapping the scan warning position to the preprocessing result to obtain a mapping image includes: Based on the correspondence between pixels and actual distances, the scanned warning position is converted into warning position pixel coordinates; Pixel mapping of the preprocessing result is performed according to the pixel coordinates of the warning position to obtain a mapping image.
6. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1, characterized in that: Get the mapping image analysis results, including: Perform pixel overlap on the mapping image and the heat source area clustering result to obtain the positional relationship between the scanning warning position and the multi-heat source point area, wherein the positional relationship includes adjacent, intersecting and containing; Based on the positional relationship, a pixel group where the scanning warning position overlaps with the multi-heat source point area is obtained to obtain an area overlap ratio; Based on the area overlap ratio, the position relationship is screened for associated areas to obtain associated locations and non-associated locations; Performing heat source target attribute anomaly identification on all associated areas in the associated position, and extracting heat source extreme value points of each associated area as abnormal target positions; Marking all scanning warning positions of the non-associated positions as abnormal target attributes, and using the scanning warning positions as abnormal target positions, marking all multi-heat source point areas of the non-associated positions as normal heat source target attributes; Based on the abnormal identification of the heat source target attribute, the abnormal identification of the target attribute and the corresponding abnormal target location, a real-time warning analysis location is obtained, and based on the normal identification of the heat source target attribute, a non-warning location is obtained; Based on the real-time warning analysis position and the non-warning position, a mapping image analysis result is obtained.
7. The radio frequency detection and positioning method combined with multimodal vision technology according to claim 1, characterized in that: Performing abnormal device determination on the second image set, acquiring abnormal devices and locating the abnormal devices, includes: Performing image division on the second image set according to the real-time warning analysis position to obtain an image division result; Inputting the image segmentation result into an abnormal device determination model, performing image abnormality determination, and obtaining abnormal values of the image segmentation; Abnormal devices are determined based on abnormal values of the image segmentation, and the positions of the abnormal devices are obtained.
8. The radio frequency detection and positioning system combined with multimodal vision technology is characterized by: The system is connected in communication with a radio frequency detection unit, a low light level night vision unit and an infrared detection unit, and the system comprises: An infrared image acquisition module, used for acquiring an image of the detection required area based on the infrared detection unit to obtain first image data; A heat source clustering module is used to preprocess the first image data, perform multi-heat source point clustering based on the preprocessing result, and obtain a heat source area clustering result; A scanning warning position acquisition module, used to scan the detection requirement area based on the radio frequency detection unit to obtain a scanning warning position; A mapping image acquisition module, used to map the scan warning position to the preprocessing result to acquire a mapping image; An image analysis module is used to use the mapping image in combination with the heat source area clustering result to perform anomaly combination analysis, obtain heat source target attributes and abnormal target locations, and obtain mapping image analysis results; A low-light-level image acquisition module is used to extract the real-time warning analysis position in the mapping image analysis result, and use the low-light-level night vision unit to collect low-light images to obtain a second image set; The abnormal device location acquisition module is used to perform abnormal device determination on the second image set, acquire abnormal devices and locate abnormal device locations.