A method for constructing panoramic photoelectric radar based on image recognition

Abstract

The present invention discloses a method for constructing a panoramic photoelectric radar based on image recognition, comprising building a photoelectric device, wherein the photoelectric device includes a scanning actuator composed of a visible light sensor, an infrared image sensor, and a laser rangefinder, a pan-tilt motor, a mounting base, a controller integrated on a connecting base, a slip ring, and an encoder. The pan-tilt motor drives the mounting base and the scanning actuator mounted thereon to perform 360-degree continuous rotation at a constant rate to collect images. The method integrates visible light, infrared image, and laser ranging data to achieve multimodal data fusion, providing more comprehensive and accurate environmental perception. The method utilizes a multi-target tracking algorithm based on Kalman filtering to overcome the field of view limitations of traditional equipment, improve the recognition, tracking, ranging, and batch number establishment functions of target objects, and enhance the system's responsiveness. Real-time display is achieved through a photoelectric radar display and control software platform, and radar information is integrated to provide more intuitive and accurate situation information, thereby improving decision-making efficiency.

Description

Technical Field

[0001] The present invention relates to the field of ship monitoring and tracking, and in particular to a method for constructing a panoramic photoelectric radar based on image recognition. Background Art

[0002] Under the new situation of economic globalization, global trade exchanges are extremely close. The total import and export volume of countries around the world, especially my country, has shown a rapid growth rate. Shipping transportation is favored by global businesses for its huge cargo volume and efficient cargo protection. Therefore, the number of ships built and the tonnage of ships by shipbuilding companies have increased year by year. The safety of ships during navigation has always been the focus of people's attention.

[0003] Traditional optoelectronic equipment is limited in operation to the fact that it can only intelligently identify and assign batch numbers to images within the current field of view. Once the hull rotates or shifts, if the target moves out of the current field of view, the equipment will not be able to continue to effectively identify, track and measure distance. This limitation is particularly prominent during maritime navigation. Due to the frequent changes in the ship's heading and attitude, when the target is within the field of view, the equipment can assign it an optoelectronic batch number based on the recognition results. However, when the hull movement causes the target to move out of the field of view, the batch number is lost. When the same target enters the field of view again, the equipment will re-assign it a new batch number. This not only leads to instability in batch number management, but also weakens its reliability.

[0004] To solve this problem, we propose a method for constructing a panoramic optoelectronic radar based on image recognition. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, which can effectively solve the problems in the background technology.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for constructing a panoramic photoelectric radar based on image recognition includes the following steps:

[0008] Assemble an optoelectronic device, comprising a scanning actuator composed of a visible light sensor, an infrared image sensor, and a laser rangefinder, a pan / tilt motor, a mounting base, a controller integrated with a connecting base, a slip ring, and an encoder, wherein the controller is used to control the pan / tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate continuously 360 degrees at a constant rate for image acquisition, the slip ring is disposed on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan / tilt motor;

[0009] During navigation, the visible light sensor, infrared image sensor, and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images, and distance data from various angles, and associate the corresponding timestamp information with the rotation angle information.

[0010] Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity;

[0011] Feature extraction algorithms are used to extract feature points from visible light and infrared images. Feature matching algorithms are used to match feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images. The aligned images are seamlessly fused to generate visible light and infrared panoramas.

[0012] Perform contrast adjustment and histogram equalization on the visible light panorama and infrared panorama. Use pixel-level graphics fusion to process the visible light image and infrared image. Add each pixel value of the visible light image and infrared image according to the weight to obtain the panoramic image.

[0013] Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information;

[0014] Assign a unique batch number based on each detected target object;

[0015] Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information of the target objects;

[0016] Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library;

[0017] The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

[0018] Preferably, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images and distance data at various angles, and associate corresponding timestamp information and rotation angle information, specifically including:

[0019] An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded.

[0020] Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with target scene point cloud data acquired by scanning the target scene with a laser rangefinder;

[0021] The first-category single-frame portraits, second-category single-frame portraits, and point cloud data obtained at different angles during one rotation are sorted and packaged into a set of data sets. The first-category single-frame portraits, second-category single-frame portraits, and point cloud data obtained at the same angle in a set of data sets are sorted and recorded as the first sub-dataset, the second sub-dataset...the nth sub-dataset, and the corresponding timestamp information and rotation angle information are associated with each sub-dataset.

[0022] Preferably, the method of using an optimized signal processing algorithm to process the collected visible light and infrared images to reduce noise interference and improve image clarity specifically includes:

[0023] A spatial Gaussian kernel and an intensity Gaussian kernel are defined. The Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

[0024] Preferably, the method comprises extracting feature points from visible light and infrared images by a feature extraction algorithm, matching feature points between images by a feature matching algorithm, calculating a transformation matrix between images based on the matched feature points, performing image alignment, and seamlessly fusing the aligned images to generate a visible light panorama and an infrared panorama, which specifically includes:

[0025] Sort the sub-data sets in a data set by timestamp information, and detect the feature points in the first category and the second category of single images in the sub-data sets using the SIFT algorithm;

[0026] Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. Based on the calculated transformation matrix, the first-category single images of the sub-datasets in a set of data sets are sequentially projected onto a reference plane and aligned. The multiple aligned first-category single images are merged into a visible light panorama. For overlapping areas, Poisson image editing is used to smooth the transition.

[0027] Based on the FLANN algorithm, the feature points in the second-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the second-category single images of the sub-datasets in a set of data sets are sequentially projected onto the reference plane and aligned. The multiple aligned second-category single images are merged into an infrared panorama. For the overlapping areas, Poisson image editing is used to achieve smooth transition.

[0028] Preferably, the contrast adjustment and histogram equalization of the fused image are performed, and the visible light image and the infrared image are processed using pixel-level graphic fusion, and each pixel value of the visible light image and the infrared image is added according to the weight, specifically including:

[0029] Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama;

[0030] Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram uniformly distributed;

[0031] Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction;

[0032] The weights of the visible light panorama and infrared panorama are defined according to application requirements, and the weighted average value of each pair of corresponding pixels in the visible light panorama and infrared panorama is calculated to perform pixel-level fusion.

[0033] Preferably, detecting the target object category and its coordinate position in the panoramic image and calculating the target object category information and coordinate information specifically includes:

[0034] Using deep learning image recognition technology, the pre-trained targets are selected based on their categories using different colored recognition boxes on the panoramic image. The target object detection bounding box distribution image and its corresponding target object category information are calculated.

[0035] Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object;

[0036] Assign a unique batch number to each target object in the panoramic image.

[0037] Preferably, the Kalman filter-based tracking algorithm analyzes the coordinate changes of target objects of the same category in different panoramic images and calculates the speed information of the target objects, specifically including:

[0038] Obtain a corresponding panoramic image based on each data set, load different panoramic images and obtain the coordinate information of the target object corresponding to the recognition frame;

[0039] Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters;

[0040] Use Kalman filter to estimate the current position of the target object, track the target object and track its motion trajectory;

[0041] According to the speed part in the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object.

[0042] Compared with the existing technology, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, which has the following beneficial effects:

[0043] By integrating visible light, infrared imagery, and laser ranging data, multimodal data fusion is achieved, providing more comprehensive and accurate environmental perception. Image optimization, including noise reduction, contrast adjustment, and histogram equalization, significantly improves image quality and subsequent image analysis and target recognition performance. Efficient image stitching and fusion technology is used to achieve precise alignment and seamless fusion of images, generating high-quality panoramic images. With the help of deep learning models, target objects can be accurately identified and classified in panoramic images and assigned unique batch numbers. The Kalman filter signal tracking algorithm is used to improve the positioning and tracking accuracy of target objects and enhance the dynamic response capability of the system. The acquired information is uploaded to the data management library and displayed in real time through the optoelectronic radar display and control software platform, allowing operators to quickly obtain key situation information.

[0044] This method integrates optoelectronic equipment, image recognition, batch number management and target detection technology to achieve efficient acquisition and intelligent recognition of 360-degree panoramic images, while ensuring the stability and accuracy of object batch numbers. This method has broad application prospects in military reconnaissance, terrain mapping, intelligent transportation and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a method for constructing a panoramic photoelectric radar based on image recognition according to the present invention;

[0046] Figure 2A flowchart of the present invention for collecting visible light images, infrared images and distance data at various angles and associating corresponding timestamp information and rotation angle information;

[0047] Figure 3 Flowchart for generating visible light panorama and infrared panorama for the present invention;

[0048] Figure 4 This is a flow chart of the present invention using pixel-level graphics fusion to process visible light images and infrared images. DETAILED DESCRIPTION

[0049] To facilitate understanding of the technical means, creative features, objectives, and effects achieved by the present invention, the present invention will be further described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0050] In order to address the shortcomings of existing technologies, such as Figure 1 As shown, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, comprising the following steps:

[0051] Assemble an optoelectronic device, comprising a scanning actuator composed of a visible light sensor, an infrared image sensor, and a laser rangefinder, a pan / tilt motor, a mounting base, a controller integrated with a connecting base, a slip ring, and an encoder, wherein the controller is used to control the pan / tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate continuously 360 degrees at a constant rate for image acquisition, the slip ring is disposed on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan / tilt motor;

[0052] During navigation, the visible light sensor, infrared image sensor, and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images, and distance data from various angles, and associate the corresponding timestamp information with the rotation angle information.

[0053] Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity;

[0054] Feature extraction algorithms are used to extract feature points from visible light and infrared images. Feature matching algorithms are used to match feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images. The aligned images are seamlessly fused to generate visible light and infrared panoramas.

[0055] Perform contrast adjustment and histogram equalization on the visible light panorama and infrared panorama. Use pixel-level graphics fusion to process the visible light image and infrared image. Add each pixel value of the visible light image and infrared image according to the weight to obtain the panoramic image.

[0056] Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information;

[0057] Assign a unique batch number based on each detected target object;

[0058] Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information of the target objects;

[0059] Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library;

[0060] The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

[0061] Specifically, such as Figure 2 As shown, the visible light sensor, infrared image sensor, and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images, and distance data from various angles, and associate the corresponding timestamp information with rotation angle information, specifically including:

[0062] An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded.

[0063] Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with target scene point cloud data acquired by scanning the target scene with a laser rangefinder;

[0064] The first-category single images, second-category single images, and point cloud data obtained at different angles during one rotation are sorted and packaged into a data set. The first-category single images, second-category single images, and point cloud data obtained at the same angle in a data set are sorted and recorded as the first sub-dataset, the second sub-dataset, ... the nth sub-dataset, and each sub-dataset is associated with the corresponding timestamp information and rotation angle information;

[0065] It should be noted that the pan-tilt motor performs continuous rotational motion, and the visible light sensor, infrared image sensor and laser rangefinder collect visible light images, infrared images and distance data at various angles. Each set of data will be associated with corresponding timestamp information and rotation angle information. By recording the timestamp information, the temporal consistency of the collected data is ensured. By recording the rotation angle information, the data collected by different sensors can be aligned to the same coordinate system, thereby improving the accuracy of the data.

[0066] Specifically, the collected visible light and infrared images are processed using optimized signal processing algorithms to reduce noise interference and improve image clarity, including:

[0067] A spatial Gaussian kernel and an intensity Gaussian kernel are defined. The Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

[0068] Specifically, such as Figure 3 As shown, feature points in visible light and infrared images are extracted through a feature extraction algorithm, and feature points between images are matched through a feature matching algorithm. Based on the matched feature points, the transformation matrix between the images is calculated, and image alignment is performed. The aligned images are seamlessly fused to generate visible light panorama and infrared panorama. Specifically, the following steps are performed:

[0069] Sort the sub-data sets in a data set by timestamp information, and detect the feature points in the first category and the second category of single images in the sub-data sets using the SIFT algorithm;

[0070] Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. Based on the calculated transformation matrix, the first-category single images of the sub-datasets in a set of data sets are sequentially projected onto a reference plane and aligned. The multiple aligned first-category single images are merged into a visible light panorama. For overlapping areas, Poisson image editing is used to smooth the transition.

[0071] Based on the FLANN algorithm, feature points in the second-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. Based on the calculated transformation matrix, the second-category single images of the sub-datasets in a set of data sets are sequentially projected onto a reference plane and aligned. The aligned second-category single images are merged into an infrared panorama. For overlapping areas, Poisson image editing is used to achieve smooth transitions. The reference plane can be a cylindrical or spherical surface.

[0072] It should be noted that Poisson image editing is based on the Poisson equation to solve the color and gradient matching problem on the image boundary; when processing, the overlapping area of ​​the two images is determined as the target area of ​​Poisson image editing; the gradient field is extracted from the non-overlapping area of ​​the image, and the boundary pixel values ​​of the overlapping area are used as the boundary conditions of the Poisson equation; the Poisson equation is solved using a numerical method to obtain the optimized pixel values ​​in the overlapping area; the solved pixel values ​​replace the original pixel values ​​in the overlapping area to achieve a smooth transition.

[0073] Specifically, such as Figure 4 As shown in the figure, the contrast of the fused image is adjusted and histogram equalization is performed. The visible light image and infrared image are processed using pixel-level graphic fusion. Each pixel value of the visible light image and infrared image is added according to the weight, specifically including:

[0074] Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama;

[0075] Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram uniformly distributed;

[0076] Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction;

[0077] Define the weights of the visible light panorama and infrared panorama according to application requirements, calculate the weighted average of each pair of corresponding pixels in the visible light panorama and infrared panorama, and perform pixel-level fusion;

[0078] It should be noted that the formula for pixel-level fusion is:

[0079]

[0080] In the formula, Indicates that the fused panoramic image is The pixel value of Indicates that the visible light panorama is The pixel value of Indicates that the infrared panorama is The pixel value of and Represent the weight parameters of the visible light panorama and infrared panorama respectively, and the weight parameters satisfy .

[0081] Specifically, detecting the target object category and its coordinate position in the panoramic image and calculating the target object category information and coordinate information specifically includes:

[0082] Using deep learning image recognition technology, the pre-trained targets are selected based on their categories using different colored recognition boxes on the panoramic image. The target object detection bounding box distribution image and its corresponding target object category information are calculated.

[0083] Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object;

[0084] Assign a unique batch number to each target object in the panoramic image;

[0085] It should be noted that the above method can realize the detection of target objects in panoramic images, the calculation of category information and coordinate information, and assign a unique batch number to each target object. The pre-trained targets are identified on the panoramic image, and different colored identification frames are used to select them based on the category of the target object. It can efficiently and accurately identify and locate multiple target objects, distinguish objects of different categories by color, make the target objects and their category information more intuitive, and better identify classified targets. The category information includes but is not limited to ships, offshore facilities, natural obstacles, and military targets.

[0086] Specifically, based on the Kalman filter tracking algorithm, the coordinate changes of the same type of target objects in different panoramic images are analyzed, and the speed information of the target objects is calculated, which specifically includes:

[0087] Obtain a corresponding panoramic image based on each data set, load different panoramic images and obtain the coordinate information of the target object corresponding to the recognition frame;

[0088] Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters;

[0089] Use the Kalman filter to estimate the current position of the target object, track the target object and its motion trajectory, and plot the tracked target object position information into a trajectory graph to intuitively display the target object's motion path;

[0090] According to the speed part of the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object;

[0091] It should be noted that this method can effectively track the motion of target objects in panoramic images and calculate their velocity information. The entire process needs to consider the distortion correction of panoramic images. The parameters of the Kalman filter need to be adjusted according to the actual application scenario to adapt to the motion characteristics of different targets and ensure the accuracy of tracking.

[0092] Based on the speed information of each target object, it is associated with the target object category information, coordinate information and batch number, and the above information is uploaded to the data management library. The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information, which can help operators quickly understand the dynamic changes in the current environment and improve decision-making efficiency.

[0093] In summary, the advantages of the present invention are as follows: this method can realize efficient capture and intelligent recognition of 360-degree panoramic images through multi-sensor fusion, and combines high-precision image processing and pixel-level fusion technology to significantly improve image quality and target recognition accuracy; adopts efficient image stitching and fusion technology to achieve precise alignment and seamless fusion of images, and generate high-quality panoramic images; with the help of deep learning models, it can accurately identify and classify target objects in panoramic images and assign them unique batch numbers; utilizes Kalman filter signal tracking algorithm to overcome the field of view limitations of traditional equipment, improve the positioning and tracking accuracy of target objects, and enhance the dynamic response capability of the system; in addition, the optoelectronic radar display and control software platform realizes real-time display of panoramic images, target objects, batch numbers, coordinate information and speed information, providing operators with intuitive and comprehensive situation information and improving decision-making efficiency.

[0094] The basic principles, main features and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention as claimed.

Claims

1. A method for constructing a panoramic photoelectric radar based on image recognition, characterized in that: The steps include: Assemble an optoelectronic device, comprising a scanning actuator composed of a visible light sensor, an infrared image sensor, and a laser rangefinder, a pan / tilt motor, a mounting base, a controller integrated with a connecting base, a slip ring, and an encoder, wherein the controller is used to control the pan / tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate continuously 360 degrees at a constant rate for image acquisition, the slip ring is disposed on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan / tilt motor; During navigation, the visible light sensor, infrared image sensor, and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images, and distance data from various angles, and associate the corresponding timestamp information with the rotation angle information. Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity; Feature extraction algorithms are used to extract feature points from visible light and infrared images. Feature matching algorithms are used to match feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images. The aligned images are seamlessly fused to generate visible light and infrared panoramas. Perform contrast adjustment and histogram equalization on the visible light panorama and infrared panorama. Use pixel-level graphics fusion to process the visible light image and infrared image. Add each pixel value of the visible light image and infrared image according to the weight to obtain the panoramic image. Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information; Assign a unique batch number based on each detected target object; Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information and prediction box position of the target object; Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library; The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

2. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 1, characterized in that: The visible light sensor, infrared image sensor, and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images, and distance data at various angles, and associate corresponding timestamp information with rotation angle information, specifically including: An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded. Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with target scene point cloud data acquired by scanning the target scene with a laser rangefinder; The first-category single-frame portraits, second-category single-frame portraits, and point cloud data obtained at different angles during one rotation are sorted and packaged into a set of data sets. The first-category single-frame portraits, second-category single-frame portraits, and point cloud data obtained at the same angle in a set of data sets are sorted and recorded as the first sub-dataset, the second sub-dataset...the nth sub-dataset, and the corresponding timestamp information and rotation angle information are associated with each sub-dataset.

3. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 2, characterized in that: The optimized signal processing algorithm is used to process the collected visible light and infrared images to reduce noise interference and improve image clarity, specifically including: A spatial Gaussian kernel and an intensity Gaussian kernel are defined. The Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

4. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 3, characterized in that: The method extracts feature points from visible light and infrared images using a feature extraction algorithm, matches feature points between images using a feature matching algorithm, calculates a transformation matrix between images based on the matched feature points, performs image alignment, and seamlessly fuses the aligned images to generate a visible light panorama and an infrared panorama. Specifically, the method includes: Sort the sub-data sets in a data set by timestamp information, and detect the feature points in the first category and the second category of single images in the sub-data sets using the SIFT algorithm; Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. Based on the calculated transformation matrix, the first-category single images of the sub-datasets in a set of data sets are sequentially projected onto a reference plane and aligned. The multiple aligned first-category single images are merged into a visible light panorama. For overlapping areas, Poisson image editing is used to smooth the transition. Based on the FLANN algorithm, the feature points in the second-category single images of adjacent sub-datasets are matched. A certain number of point pairs are randomly selected from the matched point pairs, and the homography matrix is ​​calculated. The optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the second-category single images of the sub-datasets in a set of data sets are sequentially projected onto the reference plane and aligned. The multiple aligned second-category single images are merged into an infrared panorama. For the overlapping areas, Poisson image editing is used to achieve smooth transition.

5. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 4, characterized in that: The fused image is subjected to contrast adjustment and histogram equalization. The visible light image and infrared image are processed using pixel-level graphic fusion. Each pixel value of the visible light image and infrared image is added according to the weight. Specifically, the following is performed: Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama; Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram uniformly distributed; Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction; The weights of the visible light panorama and infrared panorama are defined according to application requirements, and the weighted average value of each pair of corresponding pixels in the visible light panorama and infrared panorama is calculated to perform pixel-level fusion.

6. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 5, characterized in that: The detecting the target object category and its coordinate position in the panoramic image and calculating the target object category information and coordinate information specifically includes: Using deep learning image recognition technology, the pre-trained targets are selected based on their categories using different colored recognition boxes on the panoramic image. The target object detection bounding box distribution image and its corresponding target object category information are calculated. Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object; Assign a unique batch number to each target object in the panoramic image.

7. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 6, characterized in that: The Kalman filter tracking algorithm is based on analyzing the coordinate changes of the same type of target objects in different panoramic images and calculating the speed information of the target objects, specifically including: Obtain a corresponding panoramic image based on each data set, load different panoramic images and obtain the coordinate information of the target object corresponding to the recognition frame; Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters; Use Kalman filter to estimate the current position of the target object, track the target object and track its motion trajectory; According to the speed part in the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object.

Images