Target detection method based on multi-core DSP

Abstract

The invention discloses a target detection method based on a multi-core DSP, and aims to provide the target detection method based on the multi-core DSP, which can give consideration to both speed and performance. The specific process comprises the following five sub-steps: acquiring radar signals to obtain coarse target guiding position information, converting the guiding position information into position information of a corresponding coordinate system in an image, guiding image cutting, preprocessing a multi-core parallel image target and performing forward prediction of an identification network, and converting reverse coordinates into an inertial coordinate system. The method comprises the following steps: adding guide position information of electric signal processing on a conventional image processing method to cut an image in a targeted manner, carrying out parallel processing on the filtered cut image on a DSP (Digital Signal Processor), identifying forward prediction of a network, then obtaining a target, and converting the target to an inertial coordinate system. The method has the advantages that electrical information and optical information are organically fused, target detection of the anti-residual convolutional network on a multi-core DSP platform is achieved, speed and performance are both considered, and the method is suitable for more real-time scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of embedded systems, and more particularly to a method for object detection based on a multi-core DSP. Background Art

[0002] In recent years, with the continuous development of artificial intelligence, especially the improvement of computing power on platforms such as GPUs and FPGAs, object detection technology has also been greatly developed. However, mature object detection deep learning technologies on the market, such as tensorflow and pytorch, do not support deployment on DSP platforms, and the DSP platform is a commonly used real-time processing platform on airborne embedded devices. Therefore, it is of great significance to develop object detection methods on the DSP platform.

[0003] Object detection is an important task in the field of computer vision. Using only electronic information to process objects can obtain little effective information, and using only optical images to obtain objects will increase a lot of interference information and reduce the real-time performance of the system. Single detection sources face the pain points of slow processing speed and poor detection performance. Summary of the Invention

[0004] The present invention is to overcome the above deficiencies in the prior art and provides a method for object detection based on a multi-core DSP that can balance speed and performance.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A method for object detection based on a multi-core DSP, specifically including the following steps: (1) Obtain the electronic information of the object through electronic scanning, and calculate the longitude and latitude position information of the object through Kepler's law; obtain the optical information of the object through an optical camera, and obtain the position information of the object in the camera image pixel coordinate system through optical image processing; (2) Calibrate the conversion relationship matrix between the longitude and latitude position information obtained by the electronic information and the position information of the camera image pixel coordinate system; (3) Convert the longitude and latitude position information of the object to the position information of the image pixel coordinate system, and crop the image with the position information of the image pixel coordinate system after conversion of the longitude and latitude position information of the object required by the task as the center point to obtain a sliced image; (4) Preprocess the sliced image, obtain the region of interest ROI, and then perform anti-residual network recognition; (5) Perform a transformation from the position in the image pixel coordinate system of the recognized object to the position in the inertial coordinate system to obtain the position in the inertial coordinate system.

[0006] The present invention organically integrates electrical information and optical information, and realizes target detection by an inverse residual convolutional network on a multi-core DSP platform, taking into account both speed and performance, and is applicable to more real-time scenarios; it fully utilizes the different characteristics of electrical information and optical information, complements and unifies each other, and is of great significance for false alarm screening and operation speed in target detection; it can be applied to scenarios such as remote sensing target detection, optimize the rate problem of target detection in an embedded system, and give full play to the implementation processing performance of the multi-core DSP.

[0007] Preferably, in step (2), when calculating the conversion relationship, the longitude and latitude position information obtained by the electronic information and the position information of the target in the camera image pixel coordinate system obtained by the optical information need to unify the coordinate system, and the coordinate system is unified to the ground longitude and latitude; specifically: the electronic information coordinate system is converted to the satellite body coordinate system, the satellite body coordinate system is converted to the inertial coordinate system, and then the inertial coordinate system is converted to the geodetic coordinate system, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and the elevation information; when processing the optical image, it is converted from the position in the image pixel coordinate system to the position in the camera coordinate system, and the camera coordinate system coordinates are obtained according to the camera parameters and the installation matrix, and then are successively converted to the satellite body coordinates, inertial coordinates, and geodetic coordinates, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and the elevation information.

[0008] Preferably, in step (2), it also includes the optimization of the conversion relationship. The optimization process includes the optimization of the processing speed and the calculation method. Whether to use the optimization strategy is selected according to the matrix parameter data volume: for the optimization of the processing speed, the method of looking up a table is used to replace the commonly used fixed floating-point calculation to optimize the processing speed; for the optimization of the calculation method, when the matrix parameter data is an identity matrix or some parameters are 0, the matrix data is flattened, that is, dimensionality reduction calculation is performed, and the two-dimensional matrix is reduced to a one-dimensional array. (*, ,*) = (*,X start ,…,Xn,…X end ,*) where X start is the starting size of dimension n, Xn is the size of dimension n, and X end is the ending size of dimension n, and * represents the number of any dimension.

[0009] Preferably, when using the optimization strategy, based on the multi-core DSP system, different cores are used to process parallel tasks. After each core finishes processing, the results are transmitted to the main control core through inter-core communication for reallocation of tasks. Among them, when performing coordinate position conversion, which involves the conversion of multiple coordinate systems, multiple cores are allocated to perform parallel calculations of non-interfering operations according to the coordinate system conversion steps and conversion relationships. The results of the parallel calculations are transmitted to the main control core through inter-core communication, and the main control core then performs operations such as merging detection results, confidence ranking, and coordinate conversion to obtain the final result and then make allocations.

[0010] Preferably, in step (3), specifically: first, use electronic scanning means to detect the target position, make a rough judgment on the presence or absence of the target through the characteristics of the electrical signal, obtain the approximate longitude and latitude position of the target and convert it into the corresponding position in the image pixel coordinate system, and intercept a 1000*1000 pixel-sized slice image with this image pixel position information as the center point considering the error range.

[0011] Preferably, in step (4), the anti-residual network recognition is specifically: layout the anti-residual target detection network algorithm on the multi-core DSP, the main core performs the control logic, and the slave cores perform parallel calculations of the anti-residual network and synchronize the results to the main core, and the main core outputs the results after comprehensive calculation; among them, the common pooling process is replaced by changing the stride of the anti-residual block to change the receptive field of the feature map.

[0012] The beneficial effects of the present invention are: organically integrating electrical information and optical information, and realizing target detection by the anti-residual convolutional network on the multi-core DSP platform, taking into account both speed and performance, and being applicable to more real-time scenarios; fully applying the different characteristics of electrical information and optical information, complementing and unifying each other, which is of great significance for false alarm screening and operation speed in target detection; can be applied to scenarios such as remote sensing target detection, optimize the rate problem of target detection in the embedded system, and give full play to the implementation processing performance of the multi-core DSP. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is the flowchart of the method of the present invention; Figure 2 is the schematic diagram of the anti-residual target detection network for image detection; Figure 3 is the schematic diagram of the detection effect. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following further describes the present invention in conjunction with the drawings and specific embodiments.

[0015] As Figure 1 described in the embodiments, taking the remote sensing ship detection task as an example, the present invention specifically illustrates a method for target detection based on a multi-core DSP, which specifically includes the following steps: (1) Obtain the electronic information of the target through electronic scanning, and calculate the longitude and latitude position information of the target through Kepler's law; obtain the optical information of the target through an optical camera, and obtain the position information of the target in the camera image pixel coordinate system through optical image processing. (2) Calibrate the conversion relationship matrix between the longitude and latitude position information obtained from the electronic information and the position information in the camera image pixel coordinate system. When calculating the conversion relationship, the longitude and latitude position information obtained from the electronic information and the position information of the target in the camera image pixel coordinate system obtained from the optical information need to unify the coordinate system, and the coordinate system is unified to the ground longitude and latitude. Specifically: the electronic information coordinate system is converted to the satellite body coordinate system, the satellite body coordinate system is converted to the inertial coordinate system, and then the inertial coordinate system is converted to the geodetic coordinate system, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and the elevation information; during optical image processing, the position in the image pixel coordinate system is converted to the position in the camera coordinate system, and the camera coordinate system coordinates are obtained according to the camera parameters and the installation matrix, and then they are successively converted to the satellite body coordinates, inertial coordinates, and geodetic coordinates, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and the elevation information.

[0016] The image pixel coordinates of the target obtained through optical image processing are converted to the camera coordinate system through the camera parameters and the installation matrix. In particular, attention needs to be paid to the installation method of the camera and the object-image relationship of the camera to ensure the correctness of the corresponding relationship matrix between the image coordinates and the camera coordinates. Then, they are successively converted to the satellite body coordinates, inertial coordinates, and geodetic coordinates, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and the elevation information. In this example, since the sea surface target is detected, the influence of the elevation information is extremely small and can be ignored, and the longitude and latitude information of the target is directly calculated.

[0017] It also includes the optimization of the conversion relationship. The optimization process includes the optimization of the processing speed and the optimization of the calculation method. Whether to use the optimization strategy is selected according to the matrix parameter data volume: for the optimization of the processing speed, the method of looking up a table is used to replace the commonly used fixed floating-point calculations to optimize the processing speed; for the optimization of the calculation method, when the matrix parameter data is an identity matrix or some parameters are 0, the matrix data is flattened, that is, dimensionality reduction calculation is performed, and the two-dimensional matrix is reduced to a one-dimensional array. (*, , *) = (*, X start , …, Xn, …X end , *) Where, X start is the starting dimension of dimension n, Xn is the dimension of dimension n, X end is the ending dimension of dimension n, and * represents the number of any dimension.

[0018] In this example, multiple 3*3 matrices are flattened, such as the electronic sensor installation matrix, the camera installation matrix, etc. For example, the identity matrix [[1,0,0],[0,1,0],[0,0,1]], after flattening, a one-dimensional array [1,0,0,0,1,0,0,0,1] is obtained, reducing the calculation from two-dimensional to one-dimensional operations, reducing loop nesting, and reducing CPU overhead.

[0019] When using the optimization strategy, based on a multi-core DSP system, different cores are used to process parallel tasks, and the results after each core's processing are transmitted to the main control core through inter-core communication for reallocation of tasks. Among them, when performing coordinate position conversion, multiple coordinate system conversions are involved. According to the coordinate system conversion steps and conversion relationships, multiple cores are allocated to perform parallel calculations of non-interfering operations simultaneously. The parallel calculation results are transmitted to the main control core through inter-core communication, and the main control core then performs operations such as merging detection results, confidence ranking, and coordinate conversion to obtain the final result and then make allocations.

[0020] When performing various coordinate system conversions, various floating-point operations with large numerical values and high precision are involved. The results of commonly used fixed floating-point numbers can be calculated in advance, and the calculated results are formed into a known constant data table. The method of looking up the table is used to replace the commonly used fixed floating-point number calculations to speed up the operation speed and improve the calculation accuracy.

[0021] (3)Convert the target longitude and latitude position information to the position information in the image pixel coordinate system, and crop the image with the position information in the image pixel coordinate system after conversion of the target longitude and latitude position information required by the task as the center point to obtain a sliced image; Specifically: First, use electronic scanning means to detect the target position, make a rough judgment on the presence or absence of the target through the characteristics of the electrical signal. The target obtained by the electronic information excludes the interference of information such as clouds and shadows. After obtaining the approximate longitude and latitude position of the target, it is converted into the corresponding position in the image pixel coordinate system. Considering the error range, a 1000*1000 pixel-sized sliced image is intercepted with this image pixel position information as the center point for the next step of target detection.

[0022] (4)Preprocess the sliced image, obtain the region of interest ROI, and then perform anti-residual network recognition; The anti-residual network recognition is specifically as follows: Layout the anti-residual target detection network algorithm on a multi-core DSP. The main core performs the control logic, and the slave cores perform parallel calculations of the anti-residual network and synchronize the results to the main core. The main core outputs the results after comprehensive calculations; among them, the method of changing the stride of the anti-residual block to change the receptive field of the feature map is used to replace the commonly used pooling process, making the structure of the entire network more unified and more convenient for parallel calculation, and further facilitating the synchronous processing of the multi-core DSP system. Specifically, the sliced image is divided into blocks, and there should be partial overlap between the divided images. The scale is defined according to half of the pixel scale of the detection target, and each core processes one piece of content. Such as Figure 2 、Figure 3 As shown below, the single-core processing process is described.

[0023] The single-core processing process is as follows: the segmented image is preprocessed to obtain a region-of-interest map, scaled to a size of 416*128*3 as the network input, and undergoes 24 layers of 3*3 convolution, batch processing, and relu6 activation. Then it enters 5 anti-residual blocks. The sliding step between each anti-residual block except the first layer is set to 2, and the pooling operation is omitted to also increase the receptive field, and feature extraction and fusion are performed at different levels of the image. After output, 3*3 convolution, flattening, fully connected, and softmax are carried out. Among them, the anti-residual block consists of 1*1 convolution, 3*3 depthwise separable convolution, and 1*1 convolution.

[0024] (5) Transform the position of the recognized target from the image pixel coordinate system to the inertial coordinate system to obtain the position in the inertial coordinate system.

[0025] The present invention specifically combines the fusion processing of electrical signal radar and optical image information, and based on the hardware processing of multi-core DSP, can complete the target detection in the optoelectronic fusion mode. Specifically: obtain the radar signal to obtain the rough target guidance position information, convert the guidance position information into the position information in the corresponding coordinate system in the image, guide image cropping, multi-core parallel image target preprocessing and perform forward prediction of the recognition network, and reverse coordinate transformation to the inertial coordinate system in 5 sub-steps. The image is specifically cropped by adding the guidance position information of the electrical signal processing to the conventional image processing method, the filtered cropped image is processed in parallel on the DSP, and the target is obtained after the forward prediction of the recognition network and transformed to the inertial coordinate system.

[0026] Table 1 Comparison table of the processing speed of the guidance positioning module Content Processing speed of the guiding and positioning module Normal calculation 53ms The present invention 15ms The above table shows the comparison test results of the method proposed in the present invention and the conventional method. It can be seen from the table that the method proposed in the present invention can effectively improve the target detection speed.

Claims

1. A method for target detection based on multi-core DSP, characterized in that: The specific steps include: (1) The electronic information of the target is obtained through electronic scanning, and the longitude and latitude position information of the target is obtained by Kepler's law calculation; the optical information of the target is obtained through an optical camera, and the camera image pixel coordinate system position information of the target is obtained through optical image processing; (2) Calibrate the conversion relationship matrix between the longitude and latitude position information obtained by the electronic information and the camera image pixel coordinate system position information; (3) The target longitude and latitude position information is converted into the image pixel coordinate system position information, and the image is cropped with the image pixel coordinate system position information after the target longitude and latitude position information is converted according to the task requirements as the center point to obtain a slice image; (4) Preprocess the slice image and obtain the region of interest (ROI) for anti-residual network recognition; (5) Transform the position of the identified target from the image pixel coordinate system to the inertial coordinate system to obtain the position in the inertial coordinate system.

2. The method for target detection based on multi-core DSP according to claim 1, characterized in that: In step (2), when calculating the conversion relationship, the longitude and latitude position information obtained by the electronic information and the camera image pixel coordinate system position information of the target obtained by the optical information need to be unified into a unified coordinate system, and the coordinate system is converted to the longitude and latitude of the ground; Specifically: the electronic information coordinate system is converted to the satellite body coordinate system, the satellite body coordinate system is converted to the inertial coordinate system, and then the inertial coordinate system is converted to the earth-fixed coordinate system, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and elevation information; during optical image processing, the image pixel coordinate system position is converted to the camera coordinate system position, and the camera coordinate system coordinates are obtained according to the camera parameters and the installation matrix, and then converted to the satellite body coordinates, inertial coordinates, and earth-fixed coordinates in turn, and then the ground longitude and latitude of the target are calculated according to the earth ellipsoid equation and elevation information.

3. A method for target detection based on multi-core DSP according to claim 1 or 2, characterized in that: In step (2), the conversion relationship is also optimized. The optimization process includes the optimization of processing speed and the optimization of calculation method. Whether to use the optimization strategy is selected according to the amount of matrix parameter data: for the optimization of processing speed, the commonly used fixed floating point calculation is replaced by the table lookup method to optimize the processing speed; For the optimization of the calculation method, when the matrix parameter data is the unit matrix or some parameters are 0, the matrix data is flattened, that is, the dimension reduction calculation is performed, and the two-dimensional matrix is ​​reduced to a one-dimensional array. (*, ,*) = (*,X start ,…,Xn,…X end ,*) Among them, X start is the starting size of dimension n, Xn is the size of dimension n, X end is the ending size of dimension n, and * indicates the number of any dimension.

4. The method for target detection based on multi-core DSP according to claim 3 is characterized in that: When using the optimization strategy, based on the multi-core DSP system, different cores are used to process parallel tasks, and the results of each processing are transmitted to the main control core through inter-core communication to reallocate the tasks; among them, when performing coordinate position conversion, it involves the conversion of multiple coordinate systems. According to the coordinate system conversion steps and the conversion relationship, multiple cores are allocated to perform parallel calculations that do not affect each other, and the parallel calculation results are transmitted to the main control core through inter-core communication. The main control core then merges the detection results, sorts the confidence levels, and performs coordinate conversion processing to obtain the final results before distribution.

5. The method for target detection based on multi-core DSP according to claim 1, characterized in that In step (3), specifically: first use electronic scanning means to detect the target position, make a rough judgment on the presence or absence of the target through the characteristics of the electrical signal, obtain the approximate longitude and latitude position of the target and convert it into the corresponding image pixel coordinate system position, consider the error range and use the image pixel position information as the center point to intercept a 1000*1000 pixel slice image.

6. The method for target detection based on multi-core DSP according to claim 1, characterized in that In step (4), the anti-residual network recognition is specifically as follows: the anti-residual target detection network algorithm is arranged on the multi-core DSP, the main core performs the control logic, the slave cores calculate the anti-residual network in parallel and synchronize the results to the main core, and the main core outputs the results after comprehensive calculation; among them, the commonly used pooling processing is replaced by changing the step size of the anti-residual block to change the receptive field of the feature map.

Images