A high-precision detection method for photoelectric targets based on digital twins

Abstract

The present invention provides a high-precision photoelectric target detection method based on digital twins, comprising: capturing active and passive images respectively, performing inter-frame difference operations to obtain differential images; performing adaptive threshold segmentation on the differential images to obtain photoelectric target candidate areas; and simultaneously inputting the real photoelectric target image and each photoelectric target candidate area into a trained digital twin network target detection model. The digital twin network target detection model determines whether the photoelectric target candidate area has a photoelectric target by comparing the similarity between the photoelectric target candidate area and the real photoelectric target image. The present invention can effectively detect potential miniature cameras in complex indoor environments and accurately locate their positions. This method is the first to propose the use of the idea of ​​digital twins to detect photoelectric targets. Compared with traditional target discrimination methods, the present invention has a higher accuracy rate, the entire detection process is logically clear and rigorous, and effectively reduces the probability of false alarms.

Description

Technical Field

[0001] The present invention belongs to the field of photoelectric detection and identification technology, and specifically relates to a high-precision detection method for photoelectric targets based on digital twins. Background Art

[0002] With the rapid development of the optoelectronics and microelectronics industries, various optoelectronic imaging devices, such as video cameras, mobile phone cameras, and surveillance cameras, are widely used in everyday applications such as corporate security, industrial vision, and street surveillance. High-performance optoelectronic equipment, such as optoelectronic reconnaissance systems, optoelectronic measurement systems, and optoelectronic tracking systems, are also being used in military applications such as target reconnaissance, early warning, and tracking. These optoelectronic imaging devices have brought great convenience to people's daily lives and played a vital role in the military.

[0003] To address the aforementioned issues, it is necessary to detect cameras and determine their locations to ensure the safety of high-security venues. Specifically, the cat's eye effect can be exploited to detect cameras. This method does not rely on camera activation or wireless signal transmission, but rather focuses solely on the camera's inherent characteristics. Laser active imaging is one of the most effective detection methods in this field and has been widely used for detecting conventional cameras. Some methods can also be used for detecting indoor micro-cameras. However, the detection accuracy of micro-cameras in complex scenarios still needs to be improved. This technology faces two major challenges: First, the photoelectric targets generated by micro-cameras occupy fewer pixels in the image, resulting in relatively few extractable features, making missed detections common. Second, due to the diverse scene types in indoor environments, photoelectric targets can easily be confused with the background or reflected light spots from high-brightness objects, making it difficult to distinguish real from fake targets using only the target's texture and light intensity characteristics. Therefore, the key technical challenge in achieving high-precision detection of photoelectric targets using laser active imaging detection technology is to effectively eliminate various bright spots in the background and accurately identify the photoelectric targets, thereby reducing the probability of false alarms.

[0004] The following literature relates to detection methods for micro cameras and similar small targets:

[0005] 1. Li L, Ren J, Wang X. Fast cat-eye effect target recognition based on saliency extraction. Optics Communications, 2015, 350:33-39. This paper uses a laser imaging system to acquire active and passive images. Based on the visual attention mechanism, a multi-channel image saliency processing method is proposed to identify photoelectric targets. After processing the eight directional channels in the initial image, a directional saliency map is obtained and fused. The fused image is then thresholded and segmented to achieve photoelectric target detection. This method relies heavily on high image resolution, which carries more information but comes with the disadvantage of being time-consuming and costly.

[0006] 2. Huang J, Zhang H, Wang L, et al. Improved YOLOv3 Model for miniature camera detection. Optics & Laser Technology, 2021, 142: 107133. This paper uses a single static active laser detection system to collect images and uses the improved YOLOv3 target detection algorithm based on deep learning to perform real-time detection on the collected active images. This method uses a single-stage detection algorithm, which is slightly inferior to the two-stage detection algorithm based on candidate regions in terms of detection accuracy and positioning precision. In addition, this method is limited by the laser incident angle and detection distance, making it difficult to apply.

[0007] In addition to the methods in the above papers, patents CN 108226953 A and CN 108320303 A are based on the optical characteristics of photoelectric targets for detection. Their ability to extract features is slightly insufficient and the operation is complicated. Summary of the Invention

[0008] In response to the defects of the existing technology, the present invention provides a high-precision detection method for photoelectric targets based on digital twins, which can effectively solve the above problems.

[0009] The technical solution adopted in the present invention is as follows:

[0010] The present invention provides a high-precision photoelectric target detection method based on digital twins, comprising the following steps:

[0011] Step S1: For the same detection area, an active image and a passive image are captured respectively; wherein the active image refers to an image captured when the detection area is irradiated with laser light; and the passive image refers to an image captured when the detection area is not irradiated with laser light;

[0012] Step S2, performing inter-frame difference operation on the active image and the passive image to obtain a differential image;

[0013] Step S3, performing adaptive threshold segmentation on the differential image to segment at least one photoelectric target candidate region in the differential image;

[0014] Step S4: using the following method to identify each photoelectric target candidate area and determine whether the photoelectric target candidate area has a photoelectric target:

[0015] Step S4.1, obtaining a real photoelectric target image;

[0016] Step S4.2: Inputting the real photoelectric target image and each of the photoelectric target candidate regions into a trained digital twin network target detection model simultaneously. The digital twin network target detection model determines whether the photoelectric target candidate region has a photoelectric target by comparing the similarity between the photoelectric target candidate region and the real photoelectric target image.

[0017] Step S5 : For the photoelectric target candidate area determined to have a photoelectric target, the position of the photoelectric target candidate area is displayed in the active image and marked as the photoelectric target identified in the active image.

[0018] Preferably, step S3 is specifically as follows:

[0019] Step S3.1: Arrange the pixels in the differential image from high to low according to the grayscale value, take the pixel at the percentile, and use the grayscale value of the first percentile pixel as the brightness segmentation threshold I th ;

[0020] Step S3.2, processing the differential image, dividing the grayscale value below the brightness segmentation threshold I th Remove the pixels whose grayscale value is greater than or equal to the brightness segmentation threshold I th That is, remove low-brightness pixels and retain high-brightness pixels, thereby obtaining a processed difference image;

[0021] Step S3.3, in the processed differential image, the grayscale value of the pixel with the highest grayscale value is obtained, which is expressed as: max ;

[0022] The threshold T is obtained using the following formula:

[0023]

[0024] Wherein: m represents the difference between the maximum gray value and the lowest gray value in the gray fluctuation of the false target area in the difference image after processing, and is the brightness segmentation threshold I th After segmentation, the difference between the photoelectric target and the background area;

[0025] k is a constant;

[0026] Step S3.4: Segment the processed differential image obtained in step S3.3 using a threshold T, removing pixels with grayscale values ​​below the threshold T and retaining only pixels with grayscale values ​​greater than or equal to the brightness threshold T, thereby obtaining a segmented differential image.

[0027] Step S3.5, determining a window size; the window size is larger than a size of a single photoelectric target;

[0028] In the segmented difference image obtained in step S3.4, the window is moved according to the set step size. At each position, the image area within the window is judged to determine whether it meets the following two conditions. If so, the image area within the window is considered to be a candidate photoelectric target area;

[0029] Condition 1: The number of pixels in the image area within the window is greater than parameter N; parameter N is determined by the following formula:

[0030]

[0031] d is the detection distance;

[0032] H' is the minimum size of the detected photoelectric target;

[0033] f is the focal length used when capturing the image;

[0034] C is the pixel size;

[0035] Condition 2: The length a and width b of the minimum bounding rectangle formed by all pixels in the image area within the window satisfy the following formula:

[0036] a:b≤2:1.

[0037] Preferably, the following method is used to train the digital twin network:

[0038] Step A1: obtaining a training sample set; the training sample set includes a plurality of training samples; the training samples include a plurality of positive training samples and a plurality of negative training samples;

[0039] The positive training sample acquisition method comprises: obtaining a pair of active image positive samples and passive image positive samples with real photoelectric targets; performing an inter-frame difference operation on the active image positive samples and the passive image positive samples, and using the obtained difference image as the positive training sample;

[0040] The negative training sample acquisition method comprises: obtaining paired active image negative samples and passive image negative samples that do not have real photoelectric targets; performing inter-frame difference operation on the active image negative samples and the passive image negative samples, and using the obtained difference image as the negative training sample;

[0041] Step A2: Establish a digital twin network target detection model;

[0042] Step A3: inputting the real photoelectric target image and the training sample into the digital twin network target detection model, and the digital twin network target detection model detects the similarity between the real photoelectric target image and the training sample to obtain a similarity value;

[0043] Step A4, determining whether the similarity value exceeds a preset similarity threshold; if so, a conclusion is drawn that the training sample contains a real photoelectric target; otherwise, a conclusion is drawn that the training sample does not contain a real photoelectric target;

[0044] In step A5, the conclusion obtained in step A4 is compared with the actual situation, and the parameter weights of the digital twin network target detection model are adjusted according to the comparison results; then, the process returns to step A3, and the next training sample is used to continue training the digital twin network target detection model until the set requirements are met, thereby obtaining a trained digital twin network target detection model.

[0045] Preferably, the digital twin network target detection model adopts a lightweight convolutional neural network, and is trained using an SGD optimizer with momentum. The momentum is set to 0.9, the learning rate is set to 0.001, and it decreases to 90% of the original every 10 generations. The number of training samples for each learning is 50.

[0046] Preferably, the digital twin network target detection model includes a first network structure, a second network structure and a Sigmoid function; the first network structure and the second network structure have the same network structure and share network weights;

[0047] The real photoelectric target image is input into the first network structure, and the first network structure performs feature extraction to obtain a first image feature value;

[0048] The training sample is input into the second network structure, and feature extraction is performed by the second network structure to obtain a second image feature value;

[0049] The first image feature value and the second image feature value are input to the Sigmoid function, and the Sigmoid function outputs the similarity between the first image feature value and the second image feature value.

[0050] Preferably, the second network structure performs feature extraction on the training sample to obtain a second image feature value, specifically:

[0051] The second network structure includes a Squeeze layer, an Expand layer, a LayerNorm normalization layer, a fully connected layer and an activation layer;

[0052] The training samples are input into the Squeeze layer and subjected to 1×1 convolution for dimensionality reduction to obtain the reduced-dimensional training samples;

[0053] The dimensionality-reduced training samples are respectively input into the 1×1 convolution and 3×3 convolution in the Expand layer for parallel calculation, and the 1-1 sample features and 1-2 sample features are output;

[0054] Inputting the 1-1 sample feature and the 1-2 sample feature into the LayerNorm normalization layer respectively to obtain the 2-1 sample feature and the 2-2 sample feature respectively;

[0055] The 2-1st sample feature and the 2-2nd sample feature are respectively input into the fully connected layer to obtain the 3-1st sample feature and the 3-2nd sample feature respectively; the 3-1st sample feature and the 3-2nd sample feature are both one-dimensional vector features, respectively expressed as: sample feature F1 and sample feature F2;

[0056] Calculate the L1 norm of the difference between the sample feature F1 and the sample feature F2, then perform two full connections on the calculation results to output a second image feature value.

[0057] Preferably, in step S5, in the active image, a prediction box is used to identify the photoelectric target candidate area, and the center image coordinates (x, y) of the prediction box are output:

[0058]

[0059] in:

[0060] (x1, y1) is the coordinate of the upper left corner of the prediction box, and (x2, y2) is the coordinate of the lower right corner of the prediction box.

[0061] The high-precision photoelectric target detection method based on digital twins provided by the present invention has the following advantages:

[0062] It can effectively detect potential micro cameras in complex indoor environments and accurately locate their positions. This method proposes for the first time the use of the idea of ​​digital twins to detect photoelectric targets. Compared with traditional target discrimination methods, the invention has higher accuracy, the entire detection process is logically clear and rigorous, and effectively reduces the probability of false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the process of the high-precision photoelectric target detection method based on digital twins provided by the present invention;

[0064] Figure 2 This is a structural diagram of the digital twin network target detection model provided by the present invention;

[0065] Figure 3 This is a graph of active images collected in an indoor environment;

[0066] Figure 4 This is a passive image collected in an indoor environment;

[0067] Figure 5 Graph showing the detection results of the method of the present invention;

[0068] Figure 6 This is the test result diagram of the conventional method;

[0069] Figure 7 is the gray value distribution of the true target in the difference image;

[0070] Figure 8 is the gray value distribution of false targets in the difference image. DETAILED DESCRIPTION

[0071] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0072] The present invention is a high-precision photoelectric target detection method based on digital twins. It can effectively detect indoor miniature cameras and confirm their positions. It utilizes the cat's eye effect produced by the camera's sensor. That is, the optical system converges the incident light onto the photoelectric sensor or graticule in the focal plane of the camera. According to the principle of reversible optical path, the light beam will return along the original optical path, and the intensity of the reflected light is 2-4 orders of magnitude higher than the intensity of the diffuse light reflected by ordinary objects and background. In view of this characteristic of photoelectric targets, the laser active image and passive image are first collected and differentially calculated and preprocessed to obtain the candidate area of ​​the photoelectric target. Then, an improved residual network based on the concept of digital twins is trained to identify the target area. Finally, the identification results are analyzed and processed, the actual photoelectric target is marked in the original image, and its accurate position is output.

[0073] The high-precision photoelectric target detection method based on digital twins provided by the present invention can effectively detect potential micro cameras in complex indoor environments and accurately locate their positions. This method proposes for the first time the use of the idea of ​​digital twins to detect photoelectric targets. Compared with traditional target discrimination methods, the present invention has higher accuracy, the entire detection process is logically clear and rigorous, and the probability of false alarms is effectively reduced.

[0074] like Figure 1 As shown, the high-precision photoelectric target detection method based on digital twins provided by the present invention includes the following steps:

[0075] Step S1: For the same detection area, an active image and a passive image are captured respectively; wherein the active image refers to an image captured when the detection area is irradiated with laser light; and the passive image refers to an image captured when the detection area is not irradiated with laser light;

[0076] like Figure 3 As shown in , it is an active image collected in an indoor environment; Figure 4 As shown in FIG, a passive image collected in an indoor environment.

[0077] Step S2, performing inter-frame difference operation on the active image and the passive image to obtain a differential image;

[0078] By controlling the closing of the laser to collect active and passive images and performing inter-frame difference calculations, the influence of excessive average grayscale values ​​of the background of the active and passive images can be eliminated.

[0079] Step S3, performing adaptive threshold segmentation on the differential image to segment at least one photoelectric target candidate region in the differential image;

[0080] This step is specifically as follows:

[0081] Step S3.1: Arrange the pixels in the differential image from high to low according to the grayscale value, take the pixel at the percentile, and use the grayscale value of the first percentile pixel as the brightness segmentation threshold I th ;

[0082] Step S3.2, processing the differential image, dividing the grayscale value below the brightness segmentation threshold I th Remove the pixels whose grayscale value is greater than or equal to the brightness segmentation threshold I th That is, remove low-brightness pixels and retain high-brightness pixels, thereby obtaining a processed difference image;

[0083] Brightness segmentation threshold I th It can be expressed by the following formula:

[0084] Ith =after_data[position(Q1)]

[0085] position(Q1)=1+(n-1)×0.01

[0086] Where: n is the number of pixels in the difference image, position(Q1) is the quantile position.

[0087] Step S3.3, in the processed differential image, the grayscale value of the pixel with the highest grayscale value is obtained, which is expressed as: max ;

[0088] The threshold T is obtained using the following formula:

[0089]

[0090] Wherein: m represents the difference between the maximum gray value and the minimum gray value in the gray fluctuation of the false target area in the difference image after processing, which can be approximately regarded as the brightness segmentation threshold I th After segmentation, the difference between the photoelectric target and the background area; in the process of solving the threshold T, the difference between the photoelectric target and the background area is th The greater the difference in the regional background after threshold segmentation, the higher the weight of m, and the closer the threshold T is to the maximum grayscale value.

[0091] k is a constant;

[0092] Step S3.4: Segment the processed differential image obtained in step S3.3 using a threshold T, removing pixels with grayscale values ​​below the threshold T and retaining only pixels with grayscale values ​​greater than or equal to the brightness threshold T, thereby obtaining a segmented differential image.

[0093] Step S3.5, determining a window size; the window size is larger than a size of a single photoelectric target;

[0094] In the segmented difference image obtained in step S3.4, the window is moved according to the set step size. At each position, the image area within the window is judged to determine whether it meets the following two conditions. If so, the image area within the window is considered to be a candidate photoelectric target area;

[0095] Condition 1: The number of pixels in the image area within the window is greater than parameter N; parameter N is determined by the following formula:

[0096]

[0097] d is the detection distance;

[0098] H' is the minimum size of the detected photoelectric target;

[0099] f is the focal length used when capturing the image;

[0100] C is the pixel size;

[0101] Condition 2: The length a and width b of the minimum bounding rectangle formed by all pixels in the image area within the window satisfy the following formula:

[0102] a:b≤2:1.

[0103] The adaptive threshold segmentation of the differential image adopted by the present invention is a threshold segmentation method using the echo characteristics of the photoelectric target, and its principle is:

[0104] Based on a large amount of experimental data, the inventors proposed an adaptive threshold segmentation method based on the statistical characteristics of the echo intensity of the photoelectric target. When the camera (i.e., the photoelectric target) is irradiated by laser light, the cat's eye effect is generated due to the light saturation of the detector, resulting in a strong echo phenomenon. In the difference image after the difference operation between the active image and the passive image frame, it is manifested as a high grayscale value characteristic and the grayscale fluctuation in the target area is small, that is, the overall grayscale value of the area is close to 1 max ;like Figure 7 As shown in the figure, the grayscale value distribution of the true target in the differential image is caused by the saturation of the detector light. The high-brightness false target appears as a higher grayscale value in the differential image due to the diffuse reflection of light, but the grayscale fluctuation in the target area is large, that is, the grayscale value changes in the I th with I max Fluctuations between Figure 8 As shown in Figure 1, based on the distribution characteristics of grayscale changes of true targets and false targets in the differential image, a threshold T is introduced to segment and identify true targets and false targets. The formula for the selected threshold T is as follows:

[0105]

[0106] Then, the image area within the window is screened, that is, the shape metric judgment step is performed. The image area within the window after threshold segmentation is screened. Since the photoelectric target appears as a solid circle or a near-circle in the image, the deformed light spots with an aspect ratio exceeding 2:1 are eliminated, and small areas with too few pixels are eliminated to obtain the final photoelectric target candidate area.

[0107] Step S4: using the following method to identify each photoelectric target candidate area and determine whether the photoelectric target candidate area has a photoelectric target:

[0108] In the present invention, the photoelectric target includes but is not limited to a micro camera.

[0109] Step S4.1, obtaining a real photoelectric target image;

[0110] Step S4.2: Inputting the real photoelectric target image and each of the photoelectric target candidate regions into a trained digital twin network target detection model simultaneously. The digital twin network target detection model determines whether the photoelectric target candidate region has a photoelectric target by comparing the similarity between the photoelectric target candidate region and the real photoelectric target image.

[0111] In the present invention, the following method is used to train the digital twin network:

[0112] Step A1: obtaining a training sample set; the training sample set includes a plurality of training samples; the training samples include a plurality of positive training samples and a plurality of negative training samples;

[0113] The positive training sample acquisition method comprises: obtaining a pair of active image positive samples and passive image positive samples with real photoelectric targets; performing an inter-frame difference operation on the active image positive samples and the passive image positive samples, and using the obtained difference image as the positive training sample;

[0114] The negative training sample acquisition method comprises: obtaining paired active image negative samples and passive image negative samples that do not have real photoelectric targets; performing inter-frame difference operation on the active image negative samples and the passive image negative samples, and using the obtained difference image as the negative training sample;

[0115] For example, 6850 active images with a size of 640*480 and 6850 passive images with a size of 640*480 are captured. Each active image corresponds to one passive image.

[0116] Each active image is cropped to obtain 20*20 pixels as active image samples; correspondingly, each passive image is cropped to obtain 20*20 pixels as passive image samples.

[0117] In order to achieve better model training results, the ratio of positive samples with real photoelectric targets to negative samples with false targets is set to 1:1, and a total of 6083 image samples are cropped.

[0118] Step A2: Establish a digital twin network target detection model;

[0119] Step A3: inputting the real photoelectric target image and the training sample into the digital twin network target detection model, and the digital twin network target detection model detects the similarity between the real photoelectric target image and the training sample to obtain a similarity value;

[0120] Step A4, determining whether the similarity value exceeds a preset similarity threshold; if so, a conclusion is drawn that the training sample contains a real photoelectric target; otherwise, a conclusion is drawn that the training sample does not contain a real photoelectric target;

[0121] For example, the present invention sets the threshold for photoelectric target similarity at 0.9. Based on the concept of digital twins, the image to be identified is compared with a real photoelectric target image. If the similarity threshold exceeds 90%, the image is considered to contain a real target; otherwise, it is considered a false target. During the inference process, the photoelectric target sample only needs to be input once, eliminating the need for re-entry for each identification, significantly improving the algorithm's detection efficiency.

[0122] In step A5, the conclusion obtained in step A4 is compared with the actual situation, and the parameter weights of the digital twin network object detection model are adjusted based on the comparison results. Then, the process returns to step A3 and the digital twin network object detection model is trained again using the next training sample until the set requirements are met, thereby obtaining a trained digital twin network object detection model. The set requirements may be: the prediction accuracy of the digital twin network object detection model is greater than a certain numerical range.

[0123] In the present invention, as a specific implementation, the digital twin network target detection model adopts a lightweight convolutional neural network, which is a lightweight convolutional neural network model improved based on the digital twin idea. The lightweight convolutional neural network mainly includes SqueezeNet, ShuffvNet, GhostNet and CondenseNet, etc. The SqueezeNet lightweight convolutional neural network provided by the present invention replaces the 3×3 convolution with 1×1 convolution and reduces the number of channels of 3×3 convolution, which greatly reduces the proportion of parameters during model operation; and by post-sampling, the network can extract more information in the Feature Map to improve network accuracy.

[0124] The SGD optimizer with momentum was used for training. The momentum was set to 0.9, the learning rate was set to 0.001, and it was reduced to 90% of the original value every 10 generations. The number of training samples for each learning was 50.

[0125] As a specific embodiment, refer to Figure 2 , the digital twin network target detection model includes a first network structure, a second network structure and a Sigmoid function; the first network structure and the second network structure have the same network structure and share network weights;

[0126] The real photoelectric target image is input into the first network structure, and the first network structure performs feature extraction to obtain a first image feature value;

[0127] The training sample is input into the second network structure, and feature extraction is performed by the second network structure to obtain a second image feature value;

[0128] The first image feature value and the second image feature value are input to the Sigmoid function, and the Sigmoid function outputs the similarity between the first image feature value and the second image feature value. The Sigmoid function is used to adjust the similarity result between 0 and 1, and the similarity result represents the similarity between the real photoelectric target image and the training sample. The Sigmoid function formula is as follows:

[0129]

[0130] The second network structure extracts features from the training sample to obtain a second image feature value, specifically:

[0131] The second network structure includes a Squeeze layer, an Expand layer, a LayerNorm normalization layer, a fully connected layer and an activation layer;

[0132] The training samples are input into the Squeeze layer and subjected to 1×1 convolution for dimensionality reduction to obtain the reduced-dimensional training samples;

[0133] The dimensionality-reduced training samples are respectively input into the 1×1 convolution and 3×3 convolution in the Expand layer for parallel calculation, and the 1-1 sample features and 1-2 sample features are output;

[0134] Inputting the 1-1 sample feature and the 1-2 sample feature into the LayerNorm normalization layer respectively to obtain the 2-1 sample feature and the 2-2 sample feature respectively;

[0135] The 2-1st sample feature and the 2-2nd sample feature are respectively input into the fully connected layer to obtain the 3-1st sample feature and the 3-2nd sample feature respectively; the 3-1st sample feature and the 3-2nd sample feature are both one-dimensional vector features, respectively expressed as: sample feature F1 and sample feature F2;

[0136] The L1 norm of the difference between the sample feature F1 and the sample feature F2 is calculated, and then the calculation result is fully connected twice to obtain the output of a neuron, and output the second image feature value.

[0137] The sample feature F1 and the sample feature F2 are subtracted and then the absolute values ​​are summed, that is, the L1 norm of the difference between the sample feature F1 and the sample feature F2 is calculated.

[0138] Step S5 : For the photoelectric target candidate area determined to have a photoelectric target, the position of the photoelectric target candidate area is displayed in the active image and marked as the photoelectric target identified in the active image.

[0139] For example, in the active image, a closed square box with a width of two pixels is used to mark the candidate area.

[0140] In the active image, a prediction box is used to identify the candidate area of ​​the photoelectric target, and the center image coordinates (x, y) of the prediction box are output:

[0141]

[0142] in:

[0143] (x1, y1) is the coordinate of the upper left corner of the prediction box, and (x2, y2) is the coordinate of the lower right corner of the prediction box.

[0144] The high-precision photoelectric target detection method based on digital twins provided by the present invention can effectively detect indoor micro cameras with high precision and confirm their positions, such as Figure 5 As shown in FIG, it is the detection result diagram of the method of the present invention; Figure 6 FIG2 is a diagram showing the detection results of a conventional method. It can be seen that the present invention effectively improves the recognition accuracy of photoelectric targets.

[0145] The steps of the present invention can be summarized as follows:

[0146] S1. Image difference and preprocessing steps:

[0147] Perform inter-frame difference calculation and image preprocessing on active and passive images to obtain photoelectric target candidate areas;

[0148] S2. Digital twin network identification steps:

[0149] The photoelectric target candidate area in the differential image is identified through a deep learning network based on digital twins to screen out false targets. The idea of ​​digital twins is to use a neural network to determine the similarity between two inputs and output a similarity value.

[0150] Specifically, an image to be identified as containing a photoelectric target and an image of the target are fed into a lightweight neural network. By calculating a loss function, the two images are compared for similarity, leading to a conclusion. This method obtains specific information about whether the image to be identified contains a real photoelectric target. Compared to single neural network detection, the present invention effectively improves the accuracy of photoelectric target recognition.

[0151] S3. Photoelectric target marking steps:

[0152] The actual specific position of the optoelectronic target obtained in the above manner is displayed in the active image and marked.

[0153] The beneficial effects of the present invention are mainly reflected in:

[0154] 1. This invention proposes for the first time a solution to the problem of photoelectric target detection based on the idea of ​​digital twins. It can solve the problem of identifying photoelectric targets in relatively complex indoor backgrounds. Compared with traditional target discrimination methods, it has higher accuracy and more advantages in detection performance.

[0155] 2. The present invention is based on the characteristic that the cat's eye effect spot produced by the micro camera has a higher intensity than the diffuse reflection spot of the background and the object. The candidate area is identified by a deep learning model to achieve the purpose of high-precision detection. The requirements and operation of the optical instrument are relatively low, and the detection accuracy is more dependent on the method itself.

[0156] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A high-precision photoelectric target detection method based on digital twins, characterized in that: The following steps are involved: Step S1: For the same detection area, an active image and a passive image are captured respectively; wherein the active image refers to an image captured when the detection area is irradiated with laser light; and the passive image refers to an image captured when the detection area is not irradiated with laser light; Step S2, performing inter-frame difference operation on the active image and the passive image to obtain a differential image; Step S3, performing adaptive threshold segmentation on the differential image to segment at least one photoelectric target candidate region in the differential image; Step S4: using the following method to identify each photoelectric target candidate area and determine whether the photoelectric target candidate area has a photoelectric target: Step S4.1, obtaining a real photoelectric target image; Step S4.2: Inputting the real photoelectric target image and each of the photoelectric target candidate regions into a trained digital twin network target detection model simultaneously. The digital twin network target detection model determines whether the photoelectric target candidate region has a photoelectric target by comparing the similarity between the photoelectric target candidate region and the real photoelectric target image. Step S5 : For the photoelectric target candidate area determined to have a photoelectric target, the position of the photoelectric target candidate area is displayed in the active image and marked as the photoelectric target identified in the active image.

2. The high-precision photoelectric target detection method based on digital twin according to claim 1 is characterized in that: Step S3 is specifically as follows: Step S3.1: Arrange the pixels in the differential image from high to low according to the grayscale value, take the pixel at the percentile, and use the grayscale value of the first percentile pixel as the brightness segmentation threshold I th ; Step S3.2, processing the differential image, dividing the grayscale value below the brightness segmentation threshold I th Remove the pixels whose grayscale value is greater than or equal to the brightness segmentation threshold I th That is, remove low-brightness pixels and retain high-brightness pixels, thereby obtaining a processed difference image; Step S3.3, in the processed differential image, the grayscale value of the pixel with the highest grayscale value is obtained, which is expressed as: max ; The threshold T is obtained using the following formula: Wherein: m represents the difference between the maximum gray value and the lowest gray value in the gray fluctuation of the false target area in the difference image after processing, and is the brightness segmentation threshold I th After segmentation, the difference between the photoelectric target and the background area; k is a constant; Step S3.4: Segment the processed differential image obtained in step S3.3 using a threshold T, removing pixels with grayscale values ​​below the threshold T and retaining only pixels with grayscale values ​​greater than or equal to the brightness threshold T, thereby obtaining a segmented differential image. Step S3.5, determining a window size; the window size is larger than a size of a single photoelectric target; In the segmented difference image obtained in step S3.4, the window is moved according to the set step size. At each position, the image area within the window is judged to determine whether it meets the following two conditions. If so, the image area within the window is considered to be a candidate photoelectric target area; Condition 1: The number of pixels in the image area within the window is greater than parameter N; parameter N is determined by the following formula: d is the detection distance; H' is the minimum size of the detected photoelectric target; f is the focal length used when capturing the image; C is the pixel size; Condition 2: The length a and width b of the minimum bounding rectangle formed by all pixels in the image area within the window satisfy the following formula: a:b≤2:

1.

3. The high-precision photoelectric target detection method based on digital twin according to claim 1 is characterized in that: The digital twin network is trained using the following method: Step A1: obtaining a training sample set; the training sample set includes a plurality of training samples; the training samples include a plurality of positive training samples and a plurality of negative training samples; The positive training sample acquisition method is as follows: obtaining paired active image positive samples and passive image positive samples with real photoelectric targets; Performing an inter-frame difference operation on the active image positive sample and the passive image positive sample, and obtaining a difference image as a positive training sample; The negative training sample acquisition method comprises: obtaining paired active image negative samples and passive image negative samples that do not have real photoelectric targets; performing inter-frame difference operation on the active image negative samples and the passive image negative samples, and using the obtained difference image as the negative training sample; Step A2: Establish a digital twin network target detection model; Step A3: inputting the real photoelectric target image and the training sample into the digital twin network target detection model, and the digital twin network target detection model detects the similarity between the real photoelectric target image and the training sample to obtain a similarity value; Step A4, determining whether the similarity value exceeds a preset similarity threshold; if so, concluding that the training sample contains a real photoelectric target; Otherwise, it is concluded that the training sample does not contain a real photoelectric target; Step A5: Compare the conclusion obtained in step A4 with the actual situation, and adjust the parameter weights of the digital twin network target detection model based on the comparison results; Then return to step A3 and continue training the digital twin network target detection model using the next training sample until the set requirements are met, thereby obtaining a trained digital twin network target detection model.

4. The high-precision photoelectric target detection method based on digital twin according to claim 3 is characterized in that: The digital twin network target detection model adopts a lightweight convolutional neural network. During training, an SGD optimizer with momentum is used. The momentum is set to 0.9, the learning rate is set to 0.001, and it decreases to 90% of the original every 10 generations. The number of training samples for each learning is 50.

5. The high-precision photoelectric target detection method based on digital twin according to claim 3 is characterized in that: The digital twin network target detection model includes a first network structure, a second network structure and a Sigmoid function; the first network structure and the second network structure have the same network structure and share network weights; The real photoelectric target image is input into the first network structure, and the first network structure performs feature extraction to obtain a first image feature value; The training sample is input into the second network structure, and feature extraction is performed by the second network structure to obtain a second image feature value; The first image feature value and the second image feature value are input to the Sigmoid function, and the Sigmoid function outputs the similarity between the first image feature value and the second image feature value.

6. The high-precision photoelectric target detection method based on digital twin according to claim 5 is characterized in that: The second network structure performs feature extraction on the training sample to obtain a second image feature value, specifically: The second network structure includes a Squeeze layer, an Expand layer, a LayerNorm normalization layer, a fully connected layer and an activation layer; The training samples are input into the Squeeze layer and subjected to 1×1 convolution for dimensionality reduction to obtain the reduced-dimensional training samples; The dimensionality-reduced training samples are respectively input into the 1×1 convolution and 3×3 convolution in the Expand layer for parallel calculation, and the 1-1 sample features and 1-2 sample features are output; Inputting the 1-1 sample feature and the 1-2 sample feature into the LayerNorm normalization layer respectively to obtain the 2-1 sample feature and the 2-2 sample feature respectively; The 2-1st sample feature and the 2-2nd sample feature are respectively input into the fully connected layer to obtain the 3-1st sample feature and the 3-2nd sample feature respectively; the 3-1st sample feature and the 3-2nd sample feature are both one-dimensional vector features, respectively expressed as: sample feature F1 and sample feature F2; Calculate the L1 norm of the difference between the sample feature F1 and the sample feature F2, then perform two full connections on the calculation results to output a second image feature value.

7. The high-precision photoelectric target detection method based on digital twin according to claim 1 is characterized in that: In step S5, in the active image, a prediction box is used to identify the photoelectric target candidate area, and the center image coordinates (x, y) of the prediction box are output: in: (x1, y1) is the coordinate of the upper left corner of the prediction box, and (x2, y2) is the coordinate of the lower right corner of the prediction box.

Images