Panoramic photoelectric radar construction method based on image recognition

Abstract

The invention discloses a panoramic photoelectric radar construction method based on image recognition, and the method comprises the steps: building photoelectric equipment which comprises a scanning execution mechanism composed of a visible light sensor, an infrared image sensor and a laser range finder, a holder motor, a mounting seat, a controller integrated on a connection pedestal, a slip ring, and an encoder; a holder motor drives a mounting seat and a scanning execution mechanism mounted on the mounting seat to continuously rotate by 360 degrees at a constant rate to collect images, and the method integrates visible light, infrared images and laser ranging data to realize multi-modal data fusion and provide more comprehensive and accurate environmental perception; a multi-target tracking algorithm based on Kalman filtering is utilized, the view field limitation of a traditional device is overcome, the recognition, tracking, distance measurement and batch number establishment functions of a target object are improved, the system response capability is enhanced, real-time display is achieved through a photoelectric radar display control software platform, radar information is fused, more visual and more accurate situation information is provided, and the real-time performance of the system is improved. And decision-making efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of ship monitoring and tracking, and in particular to a method for constructing a panoramic photoelectric radar based on image recognition. Background Art

[0002] Under the new situation of economic globalization, global trade exchanges are extremely close. The total import and export volume of countries around the world, especially my country, has shown a rapid growth rate. Shipping transportation is favored by global businesses for its huge cargo volume and efficient cargo protection. Therefore, the number of ships built and the tonnage of ships by shipbuilding companies have increased year by year. The safety of ships during navigation has always been the focus of people's attention.

[0003] Traditional optoelectronic equipment is limited in operation to the fact that it can only intelligently identify images within the current field of view and assign batch numbers. Once the hull rotates or moves, if the target moves out of the current field of view, the equipment will not be able to continue to effectively identify, track and measure distance. This limitation is particularly prominent during maritime navigation. Due to the frequent changes in the ship's heading and attitude, when the target is within the field of view, the equipment can assign an optoelectronic batch number to it based on the recognition result. However, when the hull movement causes the target to move out of the field of view, the batch number is lost. When the same target enters the field of view again, the equipment will re-assign it a new batch number, which not only leads to instability in batch number management, but also weakens its reliability.

[0004] To solve this problem, we propose a method for constructing a panoramic optoelectronic radar based on image recognition. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, which can effectively solve the problems in the background technology.

[0006] To achieve the above object, the technical solution adopted by the present invention is: A method for constructing a panoramic photoelectric radar based on image recognition comprises the following steps: Build an optoelectronic device, the optoelectronic device includes a scanning actuator composed of a visible light sensor, an infrared image sensor and a laser rangefinder, and also includes a pan-tilt motor, a mounting base, a controller integrated on a connecting base, a slip ring and an encoder, the controller is used to control the pan-tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate 360 ​​degrees continuously at a constant rate to collect images, the slip ring is arranged on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan-tilt motor; During navigation, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the gimbal motor to collect visible light images, infrared images and distance data at various angles, and associate the corresponding timestamp information and rotation angle information; Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity; The feature extraction algorithm is used to extract feature points in visible light and infrared images, and the feature matching algorithm is used to match the feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images, and the aligned images are seamlessly merged to generate visible light panorama and infrared panorama; The visible light panorama and infrared panorama are subjected to contrast adjustment and histogram equalization. The visible light image and infrared image are processed by pixel-level graphic fusion. Each pixel value of the visible light image and infrared image is added according to the weight to obtain the panoramic image. Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information; Assign a unique batch number based on each detected target object; Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information of the target objects; Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library; The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

[0007] Preferably, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images and distance data at various angles, and associate corresponding timestamp information and rotation angle information, specifically including: An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded. Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with the target scene point cloud data acquired by scanning the target scene with a laser rangefinder; The first type of single images, the second type of single images and point cloud data obtained at different angles during one rotation are sorted and packaged into a data set; the first type of single images, the second type of single images and point cloud data obtained at the same angle in a data set are sorted and recorded as the first sub-dataset, the second sub-dataset...the nth sub-dataset, and each sub-dataset is associated with the corresponding timestamp information and rotation angle information.

[0008] Preferably, the method of using an optimized signal processing algorithm to process the collected visible light and infrared images to reduce noise interference and improve image clarity specifically includes: A spatial Gaussian kernel and an intensity Gaussian kernel are defined, and the Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

[0009] Preferably, the feature extraction algorithm is used to extract feature points in the visible light and infrared images, the feature matching algorithm is used to match the feature points between the images, the transformation matrix between the images is calculated based on the matched feature points, the images are aligned, and the aligned images are seamlessly merged to generate a visible light panorama and an infrared panorama, which specifically includes: The sub-data sets in a set of data sets are sorted according to the timestamp information, and the feature points in the first category single image and the second category single image in the sub-data sets are detected by the SIFT algorithm; Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the first-category single images of the sub-data sets in a set of data sets are sequentially projected onto the reference plane and aligned, and the aligned multiple first-category single images are merged into a visible light panorama. For the overlapping areas, Poisson image editing is used for smooth transition. Based on the FLANN algorithm, the feature points in the second-category single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the second-category single images of the sub-data sets in a set of data sets are sequentially projected onto the reference plane and aligned, and the aligned multiple second-category single images are merged into an infrared panorama. For the overlapping areas, Poisson image editing is used for smooth transition.

[0010] Preferably, the contrast of the fused image is adjusted and histogram equalized, and the visible light image and the infrared image are processed by pixel-level graphic fusion, and each pixel value of the visible light image and the infrared image is added according to the weight, specifically including: Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama; Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram evenly distributed; Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction; The weights of the visible light panorama and infrared panorama are defined according to application requirements, and the weighted average value is calculated for each pair of corresponding pixels in the visible light panorama and infrared panorama to perform pixel-level fusion.

[0011] Preferably, the detecting the target object category and its coordinate position in the panoramic image and calculating the target object category information and coordinate information specifically includes: Using deep learning image recognition technology, the pre-trained targets are selected based on the categories of the target objects using recognition frames of different colors on the panoramic image, and the target object detection bounding box distribution image of the panoramic image and its corresponding target object category information are calculated; Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object; Assign a unique batch number to each target object in the panoramic image.

[0012] Preferably, the Kalman filter tracking algorithm is used to analyze the coordinate changes of the same category of target objects in different panoramic images and calculate the speed information of the target objects, specifically including: Obtaining a corresponding panoramic image based on each set of data, loading different panoramic images and obtaining coordinate information of the target object corresponding to the recognition frame; Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters; Use Kalman filter to estimate the current position of the target object, track the target object and track its movement trajectory; According to the speed part in the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object.

[0013] Compared with the prior art, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, which has the following beneficial effects: By integrating visible light, infrared images and laser ranging data, multimodal data fusion is achieved to provide more comprehensive and accurate environmental perception; by optimizing image processing, including noise reduction, contrast adjustment and histogram equalization, image quality is significantly improved, and subsequent image analysis and target recognition performance are improved; by adopting efficient image stitching and fusion technology, accurate alignment and seamless fusion of images are achieved to generate high-quality panoramic images; with the help of deep learning models, target objects can be accurately identified and classified in panoramic images and given unique batch numbers; using Kalman filter signal tracking algorithm, the positioning and tracking accuracy of target objects is improved, and the dynamic response capability of the system is enhanced; the acquired information is uploaded to the data management library, and real-time display is achieved through the optoelectronic radar display and control software platform, so that operators can quickly obtain key situation information; This method integrates optoelectronic equipment, image recognition, batch number management and target detection technology to achieve efficient acquisition and intelligent recognition of 360-degree panoramic images, while ensuring the stability and accuracy of object batch numbers. This method has broad application prospects in military reconnaissance, terrain mapping, intelligent transportation and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a method for constructing a panoramic photoelectric radar based on image recognition according to the present invention; Figure 2 A flowchart of the present invention for collecting visible light images, infrared images and distance data at various angles, and associating corresponding timestamp information and rotation angle information; Figure 3 A flow chart for generating a visible light panoramic image and an infrared panoramic image for the present invention; Figure 4 The present invention is a flow chart of processing visible light images and infrared images using pixel-level graphics fusion. DETAILED DESCRIPTION

[0015] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the present invention will be further described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] In order to address the shortcomings of the existing technology, such as Figure 1 As shown, the present invention provides a method for constructing a panoramic photoelectric radar based on image recognition, comprising the following steps: Build an optoelectronic device, the optoelectronic device includes a scanning actuator composed of a visible light sensor, an infrared image sensor and a laser rangefinder, and also includes a pan-tilt motor, a mounting base, a controller integrated on a connecting base, a slip ring and an encoder, the controller is used to control the pan-tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate 360 ​​degrees continuously at a constant rate to collect images, the slip ring is arranged on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan-tilt motor; During navigation, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the gimbal motor to collect visible light images, infrared images and distance data at various angles, and associate the corresponding timestamp information and rotation angle information; Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity; The feature extraction algorithm is used to extract feature points in visible light and infrared images, and the feature matching algorithm is used to match the feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images, and the aligned images are seamlessly merged to generate visible light panorama and infrared panorama; The visible light panorama and infrared panorama are subjected to contrast adjustment and histogram equalization. The visible light image and infrared image are processed by pixel-level graphic fusion. Each pixel value of the visible light image and infrared image is added according to the weight to obtain the panoramic image. Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information; Assign a unique batch number based on each detected target object; Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information of the target objects; Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library; The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

[0017] Specifically, Figure 2 As shown, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images and distance data at various angles, and associate the corresponding timestamp information and rotation angle information, including: An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded. Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with the target scene point cloud data acquired by scanning the target scene with a laser rangefinder; The first type of single image, the second type of single image and the point cloud data obtained at different angles during one rotation are sorted and packaged into a set of data sets; the first type of single image, the second type of single image and the point cloud data obtained at the same angle in a set of data sets are sorted and recorded as the first sub-dataset, the second sub-dataset...the nth sub-dataset, and the corresponding timestamp information and rotation angle information are associated with each sub-dataset; It should be noted that through the continuous rotation of the gimbal motor, the visible light sensor, infrared image sensor and laser rangefinder collect visible light images, infrared images and distance data at various angles. Each set of data will be associated with corresponding timestamp information and rotation angle information. By recording the timestamp information, the temporal consistency of the collected data is ensured. By recording the rotation angle information, the data collected by different sensors can be aligned to the same coordinate system to improve the accuracy of the data.

[0018] Specifically, the collected visible light and infrared images are processed using an optimized signal processing algorithm to reduce noise interference and improve image clarity, including: A spatial Gaussian kernel and an intensity Gaussian kernel are defined, and the Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

[0019] Specifically, Figure 3 As shown, feature points in visible light and infrared images are extracted through feature extraction algorithms, feature points between images are matched through feature matching algorithms, transformation matrices between images are calculated based on matched feature points, and images are aligned. The aligned images are seamlessly merged to generate visible light panorama and infrared panorama, which specifically includes: The sub-data sets in a set of data sets are sorted according to the timestamp information, and the feature points in the first category single image and the second category single image in the sub-data sets are detected by the SIFT algorithm; Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the first-category single images of the sub-data sets in a set of data sets are sequentially projected onto the reference plane and aligned, and the aligned multiple first-category single images are merged into a visible light panorama. For the overlapping areas, Poisson image editing is used for smooth transition. Based on the FLANN algorithm, feature points in the second-type single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the second-type single images of the sub-data sets in a set of data sets are sequentially projected onto a reference plane and aligned, and the aligned multiple second-type single images are merged into an infrared panorama. For the overlapping areas, Poisson image editing is used for smooth transition; wherein the reference plane can be selected as a cylinder or a sphere; It should be noted that Poisson image editing is based on the Poisson equation to solve the color and gradient matching problems on the image boundary. When processing, the overlapping area of ​​the two images is determined as the target area of ​​Poisson image editing. The gradient field is extracted from the non-overlapping area of ​​the image, and the boundary pixel values ​​of the overlapping area are used as the boundary conditions of the Poisson equation. The Poisson equation is solved using a numerical method to obtain the optimized pixel values ​​in the overlapping area. The solved pixel values ​​replace the original pixel values ​​in the overlapping area to achieve a smooth transition.

[0020] Specifically, Figure 4 As shown, the contrast of the fused image is adjusted and histogram equalization is performed, and the visible light image and infrared image are processed by pixel-level graphic fusion. Each pixel value of the visible light image and the infrared image is added according to the weight, specifically including: Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama; Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram evenly distributed; Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction; Define the weights of the visible light panorama and infrared panorama according to application requirements, calculate the weighted average of each pair of corresponding pixels in the visible light panorama and infrared panorama, and perform pixel-level fusion; It should be noted that the formula for pixel-level fusion is: In the formula, Indicates that the fused panoramic image is The pixel value of Indicates that the visible light panorama is The pixel value of Indicates that the infrared panorama is The pixel value of and Respectively represent the weight parameters of the visible light panorama and infrared panorama, and the weight parameters satisfy .

[0021] Specifically, the target object category and its coordinate position in the panoramic image are detected, and the target object category information and coordinate information are calculated, which specifically includes: Using deep learning image recognition technology, the pre-trained targets are selected based on the categories of the target objects using recognition frames of different colors on the panoramic image, and the target object detection bounding box distribution image of the panoramic image and its corresponding target object category information are calculated; Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object; Assign a unique batch number to each target object in the panoramic image; It should be noted that the above method can realize the detection of target objects in panoramic images, the calculation of category information and coordinate information, and assign a unique batch number to each target object. Pre-trained targets can be identified on the panoramic image, and different colored identification boxes can be used to select them based on the category of the target object. It can efficiently and accurately identify and locate multiple target objects, distinguish objects of different categories by color, make the target objects and their category information more intuitive, and better identify classified targets. The category information includes but is not limited to ships, offshore facilities, natural obstacles, and military targets.

[0022] Specifically, based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed, and the speed information of the target objects is calculated, including: Obtaining a corresponding panoramic image based on each set of data, loading different panoramic images and obtaining coordinate information of the target object corresponding to the recognition frame; Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters; Use the Kalman filter to estimate the current position of the target object, track the target object and its motion trajectory, and draw the tracked target object position information into a trajectory map to intuitively display the motion path of the target object; According to the speed part in the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object; It should be noted that this method can effectively track the motion of target objects in panoramic images and calculate their velocity information. The whole process needs to consider the distortion correction of panoramic images. The parameters of the Kalman filter need to be adjusted according to the actual application scenarios to adapt to the motion characteristics of different targets and ensure the accuracy of tracking.

[0023] Based on the speed information of each target object, it is associated with the target object category information, coordinate information and batch number, and the above information is uploaded to the data management library. The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information, which can help operators quickly understand the dynamic changes in the current environment and improve decision-making efficiency.

[0024] In summary, the advantages of the present invention are as follows: the method can realize efficient capture and intelligent recognition of 360-degree panoramic images through multi-sensor fusion, and combines high-precision image processing and pixel-level fusion technology to significantly improve image quality and target recognition accuracy; adopts efficient image stitching and fusion technology to achieve precise alignment and seamless fusion of images, and generate high-quality panoramic images; with the help of deep learning models, it can accurately identify and classify target objects in panoramic images and assign them unique batch numbers; utilizes the Kalman filter signal tracking algorithm to overcome the field of view limitations of traditional equipment, improve the positioning and tracking accuracy of target objects, and enhance the dynamic response capability of the system; in addition, the optoelectronic radar display and control software platform realizes real-time display of panoramic images, target objects, batch numbers, coordinate information and speed information, providing operators with intuitive and comprehensive situation information and improving decision-making efficiency.

[0025] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for constructing a panoramic photoelectric radar based on image recognition, characterized in that: The steps include: Build an optoelectronic device, the optoelectronic device includes a scanning actuator composed of a visible light sensor, an infrared image sensor and a laser rangefinder, and also includes a pan-tilt motor, a mounting base, a controller integrated on a connecting base, a slip ring and an encoder, the controller is used to control the pan-tilt motor, thereby driving the mounting base and the scanning actuator mounted thereon to rotate 360 ​​degrees continuously at a constant rate to collect images, the slip ring is arranged on the mounting base and connected to the scanning actuator, and the encoder is connected to the pan-tilt motor; During navigation, the visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the gimbal motor to collect visible light images, infrared images and distance data at various angles, and associate the corresponding timestamp information and rotation angle information; Use optimized signal processing algorithms to process the collected visible light and infrared images to reduce noise interference and improve image clarity; The feature extraction algorithm is used to extract feature points in visible light and infrared images, and the feature matching algorithm is used to match the feature points between images. Based on the matched feature points, the transformation matrix between the images is calculated to align the images, and the aligned images are seamlessly merged to generate visible light panorama and infrared panorama; The visible light panorama and infrared panorama are subjected to contrast adjustment and histogram equalization. The visible light image and infrared image are processed by pixel-level graphic fusion. Each pixel value of the visible light image and infrared image is added according to the weight to obtain the panoramic image. Detect the target object category and its coordinate position in the panoramic image, and calculate the target object category information and coordinate information; Assign a unique batch number based on each detected target object; Based on the Kalman filter tracking algorithm, the coordinate changes of the same category of target objects in different panoramic images are analyzed to calculate the speed information and prediction box position of the target object; Collect the batch number information of each target object, associate it with the corresponding target object category information, coordinate information and speed information, and upload the above information to the data management library; The panoramic image, target object, batch number, coordinate information and speed information are displayed in real time on the optoelectronic radar display and control software platform as key situation information.

2. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 1, characterized in that: The visible light sensor, infrared image sensor and laser rangefinder work together and perform continuous rotation through the pan / tilt motor to collect visible light images, infrared images and distance data at various angles, and associate corresponding timestamp information and rotation angle information, specifically including: An angle increment is preset based on the gimbal motor. When the rotation angle reaches the preset angle increment value, the visible light image sensor and infrared image sensor are synchronously triggered to collect images and the laser rangefinder is used to scan the target scene, and the current timestamp information and rotation angle information are recorded. Acquire a first type of single image of the target scene acquired by a visible light sensor, acquire a second type of single image of the target scene acquired by an infrared image sensor, and align the acquired first type of single image and second type of single image with the target scene point cloud data acquired by scanning the target scene with a laser rangefinder; The first type of single images, the second type of single images and point cloud data obtained at different angles during one rotation are sorted and packaged into a data set; the first type of single images, the second type of single images and point cloud data obtained at the same angle in a data set are sorted and recorded as the first sub-dataset, the second sub-dataset...the nth sub-dataset, and each sub-dataset is associated with the corresponding timestamp information and rotation angle information.

3. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 2, characterized in that: The optimized signal processing algorithm is used to process the collected visible light and infrared images to reduce noise interference and improve image clarity, specifically including: A spatial Gaussian kernel and an intensity Gaussian kernel are defined, and the Gaussian kernel is applied to each pixel in the first-category single portrait and the second-category single portrait in each sub-dataset. The spatial weight and intensity weight of the surrounding pixels are calculated, and the calculation results are used to replace the original pixel value to perform denoising on the first-category single portrait and the second-category single portrait.

4. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 3 is characterized in that: The feature extraction algorithm is used to extract feature points in the visible light and infrared images, the feature matching algorithm is used to match the feature points between the images, the transformation matrix between the images is calculated according to the matched feature points, the images are aligned, and the aligned images are seamlessly merged to generate a visible light panorama and an infrared panorama, specifically including: The sub-data sets in a set of data sets are sorted according to the timestamp information, and the feature points in the first category of single images and the second category of single images in the sub-data sets are detected by the SIFT algorithm; Based on the FLANN algorithm, feature points in the first-category single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the first-category single images of the sub-data sets in a set of data sets are sequentially projected onto the reference plane and aligned, and the aligned multiple first-category single images are merged into a visible light panorama. For the overlapping areas, Poisson image editing is used for smooth transition. Based on the FLANN algorithm, the feature points in the second-category single images of adjacent sub-data sets are matched, a certain number of point pairs are randomly selected from the matching point pairs, the homography matrix is ​​calculated, and the optimal matrix is ​​selected through multiple iterations. According to the calculated transformation matrix, the second-category single images of the sub-data sets in a set of data sets are sequentially projected onto the reference plane and aligned, and the aligned multiple second-category single images are merged into an infrared panorama. For the overlapping areas, Poisson image editing is used for smooth transition.

5. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 4, characterized in that: The contrast adjustment and histogram equalization of the fused image are performed, and the visible light image and the infrared image are processed by pixel-level graphic fusion, and each pixel value of the visible light image and the infrared image is added according to the weight, specifically including: Use feature point matching to match the visible light panorama and the infrared panorama, apply affine transformation to correct the position deviation of the visible light panorama and the infrared panorama, and align and register the visible light panorama and the infrared panorama; Calculate the grayscale histogram of the visible light panorama and infrared panorama, calculate the cumulative distribution function, map each pixel value to a new grayscale value, and make the new histogram evenly distributed; Adjust the brightness and contrast of visible light panoramas and infrared panoramas through gamma correction; The weights of the visible light panorama and infrared panorama are defined according to application requirements, and the weighted average value is calculated for each pair of corresponding pixels in the visible light panorama and infrared panorama to perform pixel-level fusion.

6. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 5, characterized in that: The detecting the target object category and its coordinate position in the panoramic image and calculating the target object category information and coordinate information specifically includes: Using deep learning image recognition technology, the pre-trained targets are selected based on the categories of the target objects using recognition frames of different colors on the panoramic image, and the target object detection bounding box distribution image of the panoramic image and its corresponding target object category information are calculated; Match the target objects in the panoramic image with the point cloud data obtained in the corresponding target scene, and calculate the coordinate information of the point cloud data corresponding to each target object; Assign a unique batch number to each target object in the panoramic image.

7. The method for constructing a panoramic photoelectric radar based on image recognition according to claim 6, characterized in that: The Kalman filter tracking algorithm is based on analyzing the coordinate changes of the same category of target objects in different panoramic images and calculating the speed information of the target objects, specifically including: Obtaining a corresponding panoramic image based on each set of data, loading different panoramic images and obtaining coordinate information of the target object corresponding to the recognition frame; Initialize the Kalman filter, define the state vector and measurement vector of each target object, and set the filter parameters; Use Kalman filter to estimate the current position of the target object, track the target object and track its movement trajectory; According to the speed part in the state vector output by the Kalman filter, the speed change of the target object is calculated to obtain the speed information of the target object.

Images