A method and device for identifying a target in a panoramic image

Abstract

A method and device for target recognition in a panoramic image, the method comprising: obtaining a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display area of ​​the target in the panoramic image meets a preset small target condition; removing a region not including the target from a feature map of the panoramic image based on feature information of the region of interest to obtain a simplified feature map; and performing target recognition on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

Description

Technical Field

[0001] The present invention relates to information monitoring technology, and more particularly to a method and device for identifying a target in a panoramic image. Background Art

[0002] Airport surface surveillance mainly uses surface surveillance radar, Automatic Dependent Surveillance-Broadcast (ADS-B) and multi-point positioning technology. By fusing one or more data of these three technologies, the activities of aircraft and vehicles on the runway and apron can be monitored, so that controllers can fully understand and grasp the distribution and activities of various targets on the airport surface. However, these three airport surface surveillance technologies all have some shortcomings. Surface surveillance radar is expensive, cannot determine the aircraft identification code, and also has detection blind spots; Automatic Dependent Surveillance-Broadcast (ADS-B) requires aircraft and vehicles to be equipped with ADS-B transponders; multi-point positioning technology also has certain blind spots, and also requires aircraft and vehicles to be equipped with transponders. Therefore, how to monitor the airport surface is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a method and device for identifying an object in a panoramic image.

[0004] A method for identifying a target in a panoramic image, comprising: Acquire a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display region of the target in the panoramic image meets a preset small target condition; According to the feature information of the region of interest, removing the region not including the target from the feature map of the panoramic image to obtain a simplified feature map; Target recognition is performed on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

[0005] A method for identifying a target in a panoramic image, comprising: An acquisition module, configured to acquire a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display region of the target in the panoramic image meets a preset small target condition; A processing module, configured to remove the area not including the target from the feature map of the panoramic image according to the feature information of the region of interest, so as to obtain a simplified feature map; The recognition module is configured to perform target recognition on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

[0006] A storage medium stores a computer program, wherein the computer program is configured to execute the method described above when running.

[0007] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described above.

[0008] In the embodiment of the present application, the area not including the target is removed from the feature map of the panoramic image, and a large number of non-target feature blocks in the feature map are eliminated, which significantly reduces the scale of the feature map involved in the target recognition operation and effectively reduces the computational complexity of the target recognition operation. At the same time, the simplified feature map still retains the feature blocks corresponding to the target, ensuring the smooth progress of the target recognition operation. When performing target recognition on the simplified feature map, downsampling operations are avoided, thereby retaining the information of small targets to the greatest extent and effectively improving the accuracy of target recognition.

[0009] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0011] Figure 1 A schematic diagram of a flow chart of a method for identifying an object in a panoramic image provided by an embodiment of the present application; Figure 2 Another schematic diagram of the process of the method for identifying objects in a panoramic image provided by an embodiment of the present application; Figure 3 A schematic diagram of the structure of a device for identifying objects in a panoramic image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is apparent to those of ordinary skill in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0013] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0014] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0015] In the process of implementing this application, it was found that the relevant technology has the following problems, including: In related technologies, commonly used target detection algorithms need to downsample the input image to a certain size. For example, YOLO (You Only Look Once) usually uses a size of 640*640. For ultra-high-resolution panoramic stitching images, downsampling will cause serious loss of target details; if the size of the input image is increased, the consumption of computing resources will increase exponentially. In addition, currently commonly used target detection algorithms all perform indiscriminate searches on the entire input image, consuming a large amount of computing resources on meaningless background areas.

[0016] Based on the above analysis, it can be seen that the related technology has the problem that the resolution of the panoramic stitching image is too large, resulting in too much target detection calculation, and the target detection algorithm downsampling the input image causes the loss of small target information. To solve the above problems, this application proposes the following solutions, including:

[0017] Figure 1 The following is a flow chart of a method for identifying an object in a panoramic image provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0018] Step A10, acquiring a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display region of the target in the panoramic image meets a preset small target condition; The condition for a small target may be that the ratio of the display area of ​​the target to the image area in the panoramic image is smaller than a threshold; and the thresholds for different types of targets may be different.

[0019] In this step, by preliminarily finding the image area that may contain the target, there is no need to obtain the specific coordinate position and category of the target, which provides an operational basis for excluding a large number of background areas that do not contain the target.

[0020] Step A20, according to the feature information of the region of interest, removing the region not including the target from the feature map of the panoramic image to obtain a simplified feature map; In this step, based on the feature information of the region of interest, a large number of non-target feature blocks in the feature map can be removed, so that the simplified feature map is significantly smaller than the feature map before simplification, effectively reducing the amount of calculation required for the target recognition operation. Since the feature blocks corresponding to the target in the feature map are retained, the normal progress of the target recognition operation can be guaranteed.

[0021] Step A30, performing target recognition on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process; In this step, the simplified feature map is significantly smaller than the feature map before simplification, and no further downsampling operation is required. It can retain small target information to the greatest extent and effectively improve the accuracy of target recognition.

[0022] The method provided in the embodiment of the present application removes the area that does not include the target from the feature map of the panoramic image, removes a large number of non-target feature blocks in the feature map, significantly reduces the scale of the feature map involved in the target recognition operation, and effectively reduces the computational complexity of the target recognition operation. At the same time, the simplified feature map still retains the feature blocks corresponding to the target, ensuring the smooth progress of the target recognition operation. When performing target recognition on the simplified feature map, downsampling operations are avoided, thereby retaining the information of small targets to the greatest extent, effectively improving the accuracy of target recognition.

[0023] The method provided in the embodiment of the present application is described below: Among them, the region of interest (RoI) can be performed in the original image or in the feature map of the original image.

[0024] Among them, the feature map is an output in the target recognition model (such as the convolutional neural network CNN), which represents the high-dimensional feature representation of the input data after being processed by a series of convolutional layers. In image processing, the feature map can capture information such as edges, textures, colors, etc. in the image, which is crucial for subsequent image recognition and classification tasks. Each unit (or feature point) of the feature map corresponds to a local receptive field in the input image and responds to specific features (such as straight lines, curves, etc.).

[0025] The method used to obtain RoI depends on the specific requirements of the target detection or image analysis task and the method used. The specific instructions are as follows: Extract RoI from the original image. Some RoI extraction methods operate directly on the original image, such as image segmentation-based methods, saliency detection-based methods, or active contour (Snakes)-based methods. These methods usually work at the pixel level of the image and determine the RoI by identifying specific patterns or features in the image. This method is mainly suitable for tumor recognition in medical image analysis, vehicle detection in traffic monitoring, etc.

[0026] Extract RoI from the feature map of the original image. In deep learning, especially when using convolutional neural networks (CNNs) for target detection, RoI extraction is usually performed on feature maps. The feature map is the output of the original image after a series of convolutional layers and pooling layers, which contains the high-level semantic information of the image. In deep learning frameworks such as Faster R-CNN and Mask R-CNN, the feature map is first extracted using a convolutional layer, and then candidate RoIs are generated on the feature map through a region proposal network (RPN). These RoIs correspond to potential target regions in the original image. This method is suitable for scenarios where deep features need to be used for target recognition and positioning, especially when the shape, size, and position of the target vary greatly, the feature map can be used to better capture the contextual information of the target.

[0027] When the panoramic image records the monitoring data of the airport scene, the targets include vehicles, pedestrians and aircraft tail number identification codes. Step A10 includes:

[0028] Step A11, using the convolutional layer in the target recognition model to extract features from the panoramic image; Among them, the target recognition model can be a convolutional neural network (CNN).

[0029] Specifically, the convolution layer slides over the image through filters (or convolution kernels) to extract local features of the image, such as edges, textures, etc. As the network level deepens, the extracted features will gradually be combined into more abstract and advanced feature representations, providing rich information for subsequent target recognition.

[0030] Step A12: using the feature map obtained by processing the convolution layer to obtain the RoI of the panoramic image; Specifically, a region of interest of the target is acquired from the panoramic image using a lightweight model; Among them, the lightweight model is obtained by lightweight processing of a preset target recognition model, wherein the target recognition model is used to perform target recognition on a simplified feature map, wherein the lightweight processing includes retaining the recognition function of the target area in the target recognition model, and removing the function of determining the coordinate range of the target and the function of determining the category to which the target belongs in the target recognition model.

[0031] Among them, the lightweight model can use the selective search algorithm (Selective Search) to obtain the RoI of the panoramic image.

[0032] Among them, the selective search algorithm is a region proposal algorithm for target detection. It generates initial segmented regions through a graph-based image segmentation method, and then merges some small regions using a similarity calculation method to finally obtain a set of over-complete candidate regions. These candidate regions are called regions of interest, which are parts of the image that may contain target objects.

[0033] Among them, in addition to the selective search algorithm, other algorithms for region of interest (RoI) extraction are also used.

[0034] For example, in computer vision, medicine and other fields, image segmentation-based methods are widely used for RoI extraction. These methods determine the target by segmenting the image into several categories, such as using image segmentation methods based on Fuzzy C-means (FCM) and Hidden Markov Random Fields (HMRF).

[0035] For example, methods based on statistical models such as gamma distribution, gamma mixture distribution (GMD), generalized gamma distribution (GGD), etc. These models can be used for histogram-based RoI extraction, and the threshold can be adaptively calculated by establishing a mapping relationship between the required threshold and image features.

[0036] For example, saliency detection algorithms are often used to highlight RoIs in images. For example, a highly effective saliency detection algorithm based on spectral residuals is used to enhance the target region in the original image.

[0037] In this step, the region that may contain the target to be detected is obtained by obtaining RoI. The precise coordinate position and category information of the target are not required. The purpose is to reduce the background area that needs to be considered in subsequent processing and reduce the scale of the feature map involved in the target recognition operation.

[0038] In step A20, the feature information of the region of interest is a mask generated according to the feature region corresponding to the region of interest in the feature map.

[0039] The method for obtaining the simplified feature map includes: Remove non-target feature blocks from the feature map to obtain the target feature block in the feature map; Adaptively concatenate the feature blocks of the target to generate the simplified feature map.

[0040] Specifically, step A20 includes the following steps: Step A21, obtaining a mask of a feature map of the RoI; Among them, the mask is usually a binary image, in which the area corresponding to the RoI in the feature map is marked as white (or a higher gray value), while the non-RoI remains black (or a lower gray value).

[0041] Based on the results of the preliminary positioning, a feature map mask is automatically generated, which identifies the area in the image that may contain the target. Using the feature map mask to process the feature map can increase the target recognition model's attention to the target area and reduce the processing of the background area, thereby improving the accuracy and efficiency of detection.

[0042] In the object detection task, feature maps and masks can be used together to improve the accuracy and efficiency of detection. By applying a mask to the feature map, some specific elements can be shielded or selected to focus on the areas in the image that may contain the object. This method can help the model focus on the key parts of the image and exclude a large number of background areas that do not contain the object, thereby improving computational efficiency.

[0043] Step A22, obtaining a feature block after mask screening and gridding processing from the feature map; In deep learning, the process of simplifying the original feature map according to the target feature map mask can be implemented in different ways. The specific order of steps can be determined according to the actual algorithm and application scenario. The following are two common processing flows:

[0044] One is to remove non-target areas first and then perform grid segmentation.

[0045] Specifically, according to the feature map mask, the non-target area is identified and removed. This step uses the mask to block the parts of the feature map that do not belong to the area of ​​interest, and only retains the area that may contain the target; then the retained feature map area is grid-segmented. This step divides the feature map into multiple grids, each grid corresponds to a specific area in the image, which facilitates subsequent processing.

[0046] The other is to perform grid segmentation first and then remove non-target areas. Specifically, the entire feature map is grid-segmented to obtain multiple grid cells; each grid cell is then judged based on the feature map mask, and grid cells in non-target areas are removed.

[0047] When performing grid segmentation, the size and position of the grid can be dynamically adjusted based on the area identified by the mask. For example, if a region is identified as containing a target with a high probability, a finer grid division can be performed on the region to more accurately capture the characteristics of the target. After grid division, each grid corresponds to an area in the image that is considered important because they may contain targets. In this way, only these important feature blocks will be retained and used for subsequent target detection tasks.

[0048] In practical applications, which method to choose depends on the following factors: Computational resources: If computing resources are limited, it may be preferable to remove non-target areas first to reduce the amount of computation; Algorithm efficiency: Some algorithms may be better suited to meshing first, especially when specific processing strategies need to be applied to different regions; Object detection accuracy: In some cases, removing non-target areas first can improve the accuracy of object detection because it can reduce interference factors in subsequent processing.

[0049] Step A23, splicing the obtained feature blocks into a feature map to obtain a simplified feature map; The process of splicing the feature blocks after meshing involves several key steps, including the alignment, scaling and splicing strategies of the feature blocks. Among them:

[0050] When performing the alignment operation of feature blocks, it is necessary to ensure that the geometric centers or edges of the feature blocks are aligned. For example, the bilinear interpolation method can be used to achieve smooth alignment of feature blocks. When aligning, the coordinates can be adjusted to ensure that the geometric centers or edges of the feature blocks are aligned, thereby reducing irregular boundaries after splicing.

[0051] When performing the scaling operation of the feature blocks, since different feature blocks may have different sizes, they need to be scaled to ensure consistency after splicing. Scaling can be achieved through methods such as bilinear interpolation, which can maintain image quality while enlarging or reducing the image. The purpose of scaling is to make all feature blocks have the same size for easy splicing.

[0052] When determining the stitching strategy of the feature blocks, the processed feature blocks are combined into a complete feature map. This usually involves determining the position of the feature blocks in the final feature map. In an embodiment of the present application, since the targets may have different sizes and proportions, adaptive stitching can be dynamically adjusted according to the content and contextual information of the feature blocks during the stitching process to ensure that targets of different sizes can be effectively detected. For example, if a feature block contains more important target information, the stitching algorithm may give priority to the position of this block or adjust the size of other blocks to better retain the information of this feature block.

[0053] In this step, by applying a mask to the feature map, the area that may contain the target can be identified, so that the area corresponding to the mask on the feature map is retained; by removing non-target feature blocks, that is, the area identified by the mask as not containing the target, it helps to reduce the amount of calculation and focus on processing the area that may contain the target; by comprehensively considering the alignment, scaling and adaptive splicing strategies of the feature blocks, it is ensured that the streamlined feature map has the best feature fusion effect and retains the small target information to the greatest extent.

[0054] Since the size of the streamlined feature map is much smaller than that of the feature map of the panoramic image obtained by pooling, no further downsampling operation is required, which can retain the information of small targets to the greatest extent, which is especially important for detecting small targets.

[0055] In an exemplary embodiment, the step A30 includes: The target is identified by using the feature vector of the target to identify the simplified feature map, wherein the feature vector of the target is obtained by pooling the feature blocks of the region of interest in the feature map using the pooling layer of the target recognition model.

[0056] The pooling process may be a Region of Interest Pooling (RoIPooling) process; wherein RoIPooling is a technology that allows RoIs of different sizes to output fixed-size feature representations.

[0057] Wherein, step A30 specifically includes: Step A31, coordinate projection: For each region of interest, according to its coordinates in the original image (usually the coordinates of the upper left corner and the lower right corner), these coordinates are projected onto the feature map of the original image. The original is that after a series of convolution and pooling operations, the spatial size of the feature map will change, so it is necessary to convert the coordinates in the original image to the coordinates on the feature map.

[0058] Step A32: Extraction of region of interest: According to the projected coordinates, the features of the RoI are extracted on the feature map of the original image. These features are an area on the feature map, which can also be called a feature block, that is, each RoI has a rectangular area, which may contain part of one or more target objects.

[0059] Step A33, maximum pooling: Among them, Max Pooling is a pooling operation commonly used in deep learning and computer vision, especially in convolutional neural networks (CNNs). It mainly downsamples feature blocks to reduce the dimension of the data, highlight important features, and reduce computational costs.

[0060] Specifically, maximum pooling is an operation performed on the feature blocks of the extracted region of interest. It extracts the most significant features in the region and integrates these features into a fixed-length feature vector. This process can reduce the dimension of the features while retaining the most significant feature information in the region. The output after maximum pooling is a smaller feature map, which still retains the key information in the original RoI and reduces it to a smaller fixed-size feature map, thereby improving the efficiency and accuracy of tasks such as object detection.

[0061] Among them, no matter the original size of the RoI, after RoIPooling, each RoI will be converted into a feature vector of fixed size, which allows subsequent processing (such as the fully connected layer) to uniformly process inputs of different sizes.

[0062] In an exemplary embodiment, the method further comprises: After obtaining the coordinate range of the target in the simplified feature map, obtain at least 6 sample coordinates closest to the coordinate range from the preset sample coordinate data, and calculate the longitude and latitude coordinates of the target based on the at least 6 sample coordinates, where each sample coordinate has its own actual longitude and latitude coordinates.

[0063] Specifically, the image coordinates of no less than 6 sample points are manually calibrated in the original camera image, and the actual longitude and latitude coordinates of each sample point are obtained through a handheld GPS device or a GIS image map; after obtaining the real-time coordinate range of the target, the 6 points with the closest pixel distance are quickly calculated from the pre-calibrated sample coordinate data, and the least squares curve fitting is performed on these 6 points to calculate the longitude and latitude coordinates corresponding to the current coordinate range of the target.

[0064] Figure 2 Another flowchart of the method for identifying objects in a panoramic image provided by an embodiment of the present application is shown below. Figure 2 As shown, the method includes:

[0065] Step B10, obtaining a panoramic image; Specifically, video images captured by a camera are obtained, wherein the video images may be real-time or frames extracted from a video file, and the panoramic image is obtained by splicing the obtained video images.

[0066] Step B20: Obtain the RoI of the target in the panoramic image.

[0067] Step B30, using the pooling layer in the target recognition model to perform pooling processing on the RoI; Specifically, the output of the RoIPooling layer is the feature vector of each RoI, and each feature vector contains the feature information of the target in the RoI. Under the action of RoIPooling, each feature vector has a unified dimension, that is, RoIs of different sizes are converted into feature representations of fixed size so that subsequent classification and regression tasks can be processed uniformly. In addition, the feature map after maximum pooling can be flattened into feature vectors, which enter the fully connected layer for classification and regression.

[0068] Step B40: According to the mask of the feature map of the RoI, remove the non-target area in the feature map of the panoramic image to generate a simplified feature map.

[0069] There is no obvious order in which step B30 and step B40 are executed.

[0070] Step B50, using a fully connected layer in the target recognition model to process the simplified feature map according to the feature vector of the RoI to obtain a bounding box, wherein the fully connected layer is provided with a multi-task loss function; Among them, the fully connected layer is a layer in the deep network, in which each neuron is connected to all neurons in the previous layer. In the object recognition task, the fully connected layer receives the feature vector of the RoI and outputs the category and bounding box coordinates of each RoI.

[0071] In order to optimize the classification and regression tasks simultaneously, a multi-task loss function is defined, which includes classification loss (such as cross entropy loss) and regression loss (such as Smooth L1 loss or IoU loss). Among them, the classification loss is used to optimize the category prediction, while the regression loss is used to optimize the positioning of the bounding box. By minimizing the multi-task loss function, the target recognition model can simultaneously learn to identify the category of the target and accurately locate the position of the target.

[0072] The subsequent classification and regression tasks are used to determine whether each RoI contains a specific target category and accurately locate the location of these targets (i.e., bounding box regression). The classification task outputs a category probability distribution for each RoI, indicating the possibility of each category being included in the RoI; the regression task adjusts the bounding box of the RoI so that it more accurately surrounds the target object.

[0073] In practical applications, multiple RoIs may contain features of the same type of object. In these two tasks, each RoI can be analyzed independently and the category and location information can be determined for each object, even if multiple RoIs contain different instances or parts of the same type of object.

[0074] Step B60: remove the overlapping bounding boxes in the bounding boxes output by the fully connected layer to obtain the final recognition result of the target; Specifically, in the target recognition task, the same target may be detected by multiple RoIs, resulting in multiple bounding boxes overlapping. Non-Maximum Suppression (NMS) is used to remove overlapping bounding boxes and retain the best detection results. The implementation is as follows:

[0075] For each category, the bounding boxes are sorted by confidence score, and the bounding box with the highest confidence is selected as the reference box. If the intersection ratio of other bounding boxes with the reference box exceeds a set threshold, these bounding boxes are considered overlapping and are suppressed. This process is repeated until all bounding boxes are processed.

[0076] In addition to the non-maximum suppression (NMS) algorithm, there are several other algorithms that can be used to remove duplicate bounding boxes, including: Soft-NMS (Soft Non-Maximum Suppression) is an improved NMS algorithm that does not completely suppress overlapping bounding boxes, but reduces their scores. This method helps retain more information and can improve the recall rate of the model;

[0077] Diou NMS (Distance-IoU Non-Maximum Suppression) is another improved version of NMS that takes into account the diagonal distance and IoU of the bounding box. This method can more effectively handle bounding boxes of different shapes and sizes and reduce false suppression;

[0078] Confluence is a new bounding box selection and suppression method that does not rely on confidence scores to select the best bounding box or IoU to eliminate false positives. Confluence works by selecting the bounding box that is closest to all other bounding boxes in the cluster and removing neighboring boxes with high convergence.

[0079] The above algorithms provide an alternative to NMS. In practical applications, a suitable algorithm can be selected according to specific application scenarios and performance requirements.

[0080] By merging overlapping predictions, we provide a single, refined prediction for the same object, ensuring accurate and robust object detection even in complex scenes with multiple RoIs pointing to the same object.

[0081] The method provided in the embodiment of the present application removes the area that does not include the target from the feature map of the panoramic image, removes a large number of non-target feature blocks in the feature map, significantly reduces the scale of the feature map involved in the target recognition operation, and effectively reduces the computational complexity of the target recognition operation. At the same time, the streamlined feature map still retains the feature blocks corresponding to the target, ensuring the smooth progress of the target recognition operation. When performing target recognition on the streamlined feature map, downsampling operations are avoided, thereby retaining the information of small targets to the greatest extent, effectively improving the accuracy of target recognition. In addition, from image acquisition to feature extraction, and then to RoI generation and processing, each step provides indispensable information support for the final target recognition. With the help of multi-task loss functions and NMS and other technologies, targets can be accurately identified and located, achieving efficient and accurate target recognition functions.

[0082] Figure 3 This is a schematic diagram of the structure of the device for identifying objects in a panoramic image provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0083] An acquisition module 301 is configured to acquire an area of ​​interest of a target to be detected from a pre-acquired panoramic image, wherein a display area of ​​the target in the panoramic image meets a preset small target condition; The processing module 302 is configured to remove the area not including the target from the feature map of the panoramic image according to the feature information of the region of interest, so as to obtain a simplified feature map; The recognition module 303 is configured to perform target recognition on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

[0084] The device provided in the embodiment of the present application removes the area that does not include the target from the feature map of the panoramic image, removes a large number of non-target feature blocks in the feature map, significantly reduces the scale of the feature map involved in the target recognition operation, and effectively reduces the computational complexity of the target recognition operation. At the same time, the simplified feature map still retains the feature blocks corresponding to the target, ensuring the smooth progress of the target recognition operation. When performing target recognition on the simplified feature map, downsampling operations are avoided, thereby retaining the information of small targets to the greatest extent, effectively improving the accuracy of target recognition.

[0085] An embodiment of the present application provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the method described above when running.

[0086] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described above.

[0087] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term "computer storage medium" includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A method for identifying a target in a panoramic image, characterized in that: include: Acquire a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display region of the target in the panoramic image meets a preset small target condition; According to the feature information of the region of interest, removing the region not including the target from the feature map of the panoramic image to obtain a simplified feature map; Target recognition is performed on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

2. The method according to claim 1, characterized in that The feature information of the region of interest is a mask generated according to the feature region corresponding to the region of interest in the feature map.

3. The method according to claim 1, characterized in that The method for obtaining the simplified feature map includes: Remove non-target feature blocks from the feature map to obtain the target feature block in the feature map; Adaptively concatenate the feature blocks of the target to generate the simplified feature map.

4. The method according to claim 1, characterized in that: The step of acquiring a region of interest of a target to be detected from a pre-acquired panoramic image includes: Acquire a region of interest of the target from the panoramic image using a lightweight model; Among them, the lightweight model is obtained by lightweight processing of a preset target recognition model, wherein the target recognition model is used to perform target recognition on a simplified feature map, wherein the lightweight processing includes retaining the recognition function of the target area in the target recognition model, and removing the function of determining the coordinate range of the target and the function of determining the category to which the target belongs in the target recognition model.

5. The method according to claim 4, characterized in that The target recognition on the simplified feature map includes: The target is identified by using the feature vector of the target to identify the simplified feature map, wherein the feature vector of the target is obtained by pooling the feature blocks of the region of interest in the feature map using the pooling layer of the target recognition model.

6. The method according to any one of claims 1 to 5, characterized in that: When the panoramic image records the monitoring data of the airport scene, the targets include vehicles, pedestrians and tail number identification codes of aircraft.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: After obtaining the coordinate range of the target in the simplified feature map, obtain at least 6 sample coordinates closest to the coordinate range from the preset sample coordinate data, and calculate the longitude and latitude coordinates of the target based on the at least 6 sample coordinates, where each sample coordinate has its own actual longitude and latitude coordinates.

8. A method for identifying a target in a panoramic image, characterized in that: include: An acquisition module, configured to acquire a region of interest of a target to be detected from a pre-acquired panoramic image, wherein a display region of the target in the panoramic image meets a preset small target condition; A processing module, configured to remove the area not including the target from the feature map of the panoramic image according to the feature information of the region of interest, so as to obtain a simplified feature map; The recognition module is configured to perform target recognition on the simplified feature map, wherein no downsampling operation is performed on the simplified feature map during the target recognition process.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

Images