Target detection method and device of panoramic image, computer device and storage medium

Abstract

The application relates to a panoramic image target detection method and device, computer equipment and a storage medium. The application obtains a panoramic image to be detected, extracts a target detection candidate region of the panoramic image to be detected, performs convolution processing on the target detection candidate region through a preset target deformation adaptive convolution operator, obtains a boundary box corresponding to the target detection candidate region, the boundary box comprises a cyclic boundary box used for representing a boundary box with a coordinate range exceeding a boundary of the panoramic image to be detected, and a preset superposition boundary filtering algorithm is used for filtering the boundary box to obtain a target detection result corresponding to the panoramic image to be detected. When the panoramic image target detection is performed, the panoramic image does not need to be spliced, repeated contents of an input image can be effectively reduced, and the detection efficiency of target detection is improved.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to a method, apparatus, computer device, and storage medium for target detection of panoramic images. Background Technology

[0002] With the development of artificial intelligence technology, computer vision technology has been increasingly widely applied. Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to establish artificial intelligence systems capable of extracting 'information' from images or multidimensional data. Panoramic image target detection is one of the research objects of computer vision. Panoramic image target detection, also called target extraction, is an image segmentation method based on the geometric and statistical features of targets. Through panoramic image target detection, both the category and location information of targets in the image can be obtained simultaneously.

[0003] A panoramic image is a special type of image, typically with an aspect ratio of 2:1, and is composed of multiple images stitched together. It is created using a latitude-longitude unfolding method, where the width is latitude (0-2π) and the height is longitude (0-π). Therefore, it can record information across 360 degrees horizontally and 180 degrees vertically. Currently, object detection in panoramic images is generally performed on the unfolded planar image. This means that some objects in a panoramic image may be segmented to the left or right sides of the horizontal plane, making it impossible to detect them as the same object, thus affecting the accuracy of object detection in panoramic images.

[0004] Currently, to address the issue that some objects in panoramic images are segmented into the left and right sides of the horizontal direction, making it impossible to detect them as the same object, a common approach is to crop and stitch the left half of the panoramic image to the right and the right half to the left, using the stitched image as input to the detection algorithm. However, due to the stitching of the input image, a large amount of repetitive content is included, making the calculation process of the model detection more complex and affecting the efficiency of object detection in panoramic images. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for target detection of panoramic images that can simplify the calculation process of model detection and thus improve the target detection efficiency of panoramic images, in order to address the above-mentioned technical problems.

[0006] A method for target detection in panoramic images, the method comprising:

[0007] Acquire the panoramic image to be detected;

[0008] Extract the target detection candidate region from the panoramic image to be detected;

[0009] The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0010] The bounding box is filtered by a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0011] In one embodiment, the target detection candidate region is convolved using a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the non-cyclic bounding box corresponding to the target detection candidate region, including:

[0012] The panoramic image features of the target detection candidate region are extracted using the convolutional neural network.

[0013] Based on the panoramic image features, non-boundary targets in the panoramic image to be detected are determined, and non-cyclic bounding boxes corresponding to the non-boundary targets are obtained.

[0014] In one embodiment, the target detection candidate region is convolved using a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the cyclic bounding box corresponding to the target detection candidate region, including:

[0015] Panoramic image features of the target detection candidate region are extracted using a preset target deformation adaptive convolution operator;

[0016] Based on the features of the panoramic image, the target at the boundary position in the panoramic image to be detected is determined, and the detected target is obtained;

[0017] Based on the panoramic image features, target attributes between the first detection target and the second detection target are identified, where the first detection target and the second detection target are detection targets in relative positions in the panoramic image;

[0018] When the target attribute indicates that the first detection target and the second detection target are the same detection target, the corresponding circular bounding boxes of the first detection target and the second detection target are obtained.

[0019] In one embodiment, filtering the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected includes:

[0020] Obtain the confidence level corresponding to the bounding box;

[0021] The bounding boxes are sorted according to the confidence level to obtain a bounding box list;

[0022] Add the bounding box with the highest confidence to the target detection result list, remove the bounding box with the highest confidence from the bounding box list, and update the bounding box list;

[0023] Obtain the interaction ratio between the bounding box with the highest confidence level and the bounding boxes in the updated bounding box list;

[0024] Remove bounding boxes from the bounding box list whose interaction ratio is greater than a preset interaction ratio threshold;

[0025] The steps are: return to select the bounding box with the highest confidence as the target detection result, delete the bounding box with the highest confidence from the bounding box list, and update the bounding box list;

[0026] When the updated bounding box list is empty, output the latest list of target detection results.

[0027] In one embodiment, before obtaining the confidence level corresponding to the bounding box, the method further includes:

[0028] Identify the position of the circular bounding box within the bounding box;

[0029] Based on the position of the cyclic bounding box, the cyclic bounding box of any boundary in the panoramic image to be detected is mapped to the boundary corresponding to the boundary where the cyclic bounding box is located.

[0030] In one embodiment, before obtaining the interaction ratio between the bounding box with the highest confidence level and the bounding boxes in the updated bounding box list, the method further includes:

[0031] Modify the calculation rules for the interaction ratio between bounding boxes to make the calculation rules applicable to cyclic bounding boxes.

[0032] In one embodiment, extracting the target detection candidate region of the panoramic image to be detected includes:

[0033] The target detection candidate region of the panoramic image to be detected is extracted by selecting a search algorithm.

[0034] A target detection device for panoramic images, the device comprising:

[0035] The data acquisition module is used to acquire the panoramic image to be detected;

[0036] The region extraction module is used to extract the target detection candidate region of the panoramic image to be detected;

[0037] The convolution processing module is used to perform convolution processing on the target detection candidate region through a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0038] The target detection module is used to filter the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0039] In one embodiment, the convolution processing module is specifically used for:

[0040] Panoramic image features of the target detection candidate region are extracted using a preset target deformation adaptive convolution operator;

[0041] Based on the panoramic image features, non-boundary targets in the panoramic image to be detected are determined, and non-cyclic bounding boxes corresponding to the non-boundary targets are obtained.

[0042] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0043] Acquire the panoramic image to be detected;

[0044] Extract the target detection candidate region from the panoramic image to be detected;

[0045] The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0046] The bounding box is filtered by a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0047] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0048] Acquire the panoramic image to be detected;

[0049] Extract the target detection candidate region from the panoramic image to be detected;

[0050] The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0051] The bounding box is filtered by a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0052] The aforementioned panoramic image target detection method, apparatus, computer equipment, and storage medium acquire a panoramic image to be detected; extract target detection candidate regions from the panoramic image; perform convolution processing on the target detection candidate regions using a preset target deformation adaptive convolution operator to obtain bounding boxes corresponding to the target detection candidate regions. These bounding boxes include cyclic bounding boxes, which represent bounding boxes whose coordinate range exceeds the boundary of the panoramic image to be detected; and filter the bounding boxes using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected. In this application, when detecting panoramic images, a preset target deformation adaptive convolution operator is used to extract non-cyclic and cyclic bounding boxes corresponding to the target detection candidate regions. Cyclic bounding boxes represent the detection targets located at relative boundary positions in the panoramic image to be detected. Then, a preset overlay border filtering algorithm is used to filter the bounding boxes to obtain the final target detection result. This eliminates the need for panoramic image stitching, effectively reducing duplicate content in the input image and thus improving the detection efficiency of the target. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of an input image stitched together from left to right in one embodiment;

[0054] Figure 2 This is a schematic diagram of the input image and bounding box after left and right stitching in one embodiment;

[0055] Figure 3 This is an application environment diagram of a target detection method for panoramic images in one embodiment;

[0056] Figure 4 This is a flowchart illustrating a target detection method for panoramic images in one embodiment;

[0057] Figure 5 This is a flowchart illustrating the step of extracting a non-cyclic bounding box in one embodiment;

[0058] Figure 6 This is a flowchart illustrating the step of extracting the circular bounding box in one embodiment;

[0059] Figure 7 This is a flowchart illustrating the steps for obtaining target detection results in one embodiment;

[0060] Figure 8 This is a structural block diagram of a target detection device for panoramic images in one embodiment;

[0061] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] The applicant discovered that because some objects in panoramic images are segmented into the left and right sides of the horizontal axis, they cannot be detected as the same object. Therefore, current methods for object detection in panoramic images generally require cropping and stitching the left half of the horizontal image to the right and the right half to the left as input to the detection algorithm. After obtaining the detection output, duplicate bounding boxes for the same object are eliminated, and the coordinates are mapped back to the original image. This yields the final object detection result for the panoramic image. For details, please refer to [reference needed]. Figure 1 and Figure 2 , Figure 1 and Figure 2 This diagram illustrates a left-right stitched input image and the circular bounding box of the target (house). When performing target detection on panoramic images using this method, the input image contains a large amount of repetitive content, making the model's detection computation process complex and affecting the efficiency of target detection in panoramic images. To address this issue, the applicant proposes the target detection method described in this application.

[0064] The target detection method for panoramic images provided in this application can be applied to, for example... Figure 3In the application environment shown, terminal 302 communicates with server 304 via a network. When the data processing personnel on terminal 302 need to detect targets in a panoramic image, they can send the panoramic image to be detected to server 304, which then performs target detection on the submitted panoramic image. Server 304 acquires the panoramic image to be detected; extracts target detection candidate regions from the panoramic image; performs convolution processing on the target detection candidate regions using a convolutional neural network containing a preset target deformation adaptive convolution operator, obtaining the bounding boxes corresponding to the target detection candidate regions. The bounding boxes include cyclic bounding boxes, which represent bounding boxes whose coordinate range exceeds the boundary of the panoramic image to be detected; and filters the bounding boxes using a preset overlay bounding box filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected. Terminal 302 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 304 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0065] In one embodiment, such as Figure 4 As shown, a target detection method for panoramic images is provided, which can be applied to... Figure 5 Taking a 304 error on a server as an example, the steps are as follows:

[0066] Step 401: Obtain the panoramic image to be detected.

[0067] Step 403: Extract the target detection candidate region of the panoramic image to be detected.

[0068] Panoramic images are a special type of image, typically with an aspect ratio of 2:1, and are composed of multiple images stitched together. Following the latitude-longitude unfolding method, the width of the image is latitude (0-2π), and the height is longitude (0-π). Therefore, it can record all information across 360 degrees horizontally and 180 degrees vertically. Currently, when performing object detection on panoramic images, some objects are segmented to the left and right sides of the horizontal direction, making it impossible to detect them as the same object, thus affecting the accuracy of object detection in panoramic images. The object detection method for panoramic images described in this application can be used to achieve object detection for panoramic images. Candidate regions, or bounding boxes, refer to a series of regions on the input image that may contain objects, generated through some method. Currently, image segmentation and region growing techniques are mainly used to obtain candidate regions. Region growing (merging) is mainly due to the local regional similarity (color, texture, etc.) of objects in the detected image. The development of object recognition and image segmentation technologies further promotes the effective extraction of information from images.

[0069] Specifically, when terminal 302 needs to perform target detection on a panoramic image, it can submit the panoramic image to be detected to server 304 so that server 304 can perform target detection on the panoramic image to determine the target and its location within the panoramic image. Server 304 receives the panoramic image to be detected and first performs early processing of the target detection process. A series of regions that may contain objects are generated on the input panoramic image to be detected, that is, target detection candidate regions that may contain the target are extracted from the panoramic image to be detected and used as the initial data for the target detection process.

[0070] Step 405: Convolve the target detection candidate region using a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0071] In this context, operators are the basic units of neural network computation, and convolution is a frequently used operation in image processing. It enhances the features of the original signal and reduces noise. The pre-defined target deformation-adaptive convolution operator refers to the modification of existing convolutional neural network models for object detection, replacing some convolution operators with those adaptable to target deformation, such as deformable convolution, equirectangular projection convolution, and spherical convolution. These pre-defined target deformation-adaptive convolution operators are obtained through training with panoramic images. A non-cyclic bounding box refers to a bounding box whose coordinates are entirely within the boundary of the panoramic image to be detected. A cyclic bounding box refers to a bounding box whose coordinates are partially outside the boundary of the panoramic image to be detected. For panoramic images, current object detection algorithms generally perform object detection through image stitching. For example, the left half of the panoramic image can be cropped and stitched to the right, and the right half can be cropped and stitched to the left, serving as input to the detection algorithm. After obtaining the detection output, duplicate bounding boxes for the same target are eliminated, and the coordinates are mapped back to the original image. However, at this point, some bounding boxes may extend beyond the boundaries. Negative coordinates indicate that the left side extends beyond the boundary, while coordinates larger than the image width indicate that the right side extends beyond the boundary. In panoramic images, these bounding boxes are meaningful, indicating that the target exists partially on the other side of the image. These target bounding boxes with coordinates extending beyond the boundaries of the panoramic image to be detected are called recurrent bounding boxes. The convolutional neural network used here is not limited; specific implementations can be achieved using neural network models such as Faster R-CNN, SSD, YOLO, RetinaNet, and FCOS.

[0072] This application specifically employs a convolutional neural network (CNN) for object detection, enabling detection in unstitched panoramic images. During object detection, the detection model needs to extract features from both sides of the image and determine if they represent the same target. Traditional CNNs are less effective at handling this situation. Therefore, this application replaces some traditional convolution operators with a pre-defined target deformation adaptive convolution operator to construct a convolutional model better suited to panoramic images. The pre-defined target deformation adaptive convolution operator provides better adaptation to target deformation in panoramic images. By performing convolution processing on candidate regions for target detection at the boundaries of the panoramic image using this operator, it can effectively determine whether targets on both sides of the panoramic image are the same target and output a corresponding cyclic bounding box for the same target. For non-cyclic bounding boxes, other conventional object detection convolution operators of the CNN can be used for detection.

[0073] Step 407: Filter the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0074] Specifically, since the predicted bounding boxes obtained in step 405 may have overlapping portions, it is necessary to retain the predictions with the largest intersection-union ratio (IU) and remove those with non-large IUs. Therefore, a preset overlay bounding box filtering algorithm can be used to filter the bounding boxes. This algorithm can be a Non-Maximum Suppression (NMS) algorithm, or an improved version of the traditional NMS algorithm such as Soft-NMS or Softer-NMS. In one embodiment, the solution of this application can use a preset overlay bounding box filtering algorithm to filter both acyclic and cyclic bounding boxes, thereby obtaining the target detection result corresponding to the panoramic image to be detected.

[0075] The aforementioned panoramic image target detection method involves: acquiring the panoramic image to be detected; extracting target detection candidate regions from the panoramic image; performing convolution processing on the target detection candidate regions using a preset target deformation adaptive convolution operator to obtain bounding boxes corresponding to the target detection candidate regions. These bounding boxes include cyclic bounding boxes, which represent bounding boxes whose coordinate range exceeds the boundary of the panoramic image to be detected; and filtering the bounding boxes using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected. In this application, when detecting panoramic images, a preset target deformation adaptive convolution operator is used to extract non-cyclic and cyclic bounding boxes corresponding to the target detection candidate regions. Non-cyclic bounding boxes represent the detection targets located at relative boundary positions in the panoramic image to be detected. Then, a preset overlay border filtering algorithm is used to filter the bounding boxes to obtain the final target detection result. This eliminates the need for panoramic image stitching, effectively reducing duplicate content in the input image and thus improving the detection efficiency of the target.

[0076] In one embodiment, such as Figure 5 As shown, step 405 includes:

[0077] Step 502: Extract panoramic image features of the target detection candidate region using a convolutional neural network.

[0078] Step 504: Determine the non-boundary location targets in the panoramic image to be detected based on the panoramic image features, and obtain the non-cyclic bounding boxes corresponding to the non-boundary location targets.

[0079] Specifically, for the detection of non-cyclic bounding boxes, convolution calculations can be performed using common convolution operators in convolutional neural networks to extract the corresponding panoramic image features. In one embodiment, the panoramic image features may include the coordinate positions of the currently detected target detection candidate region within the panoramic image to be detected. Thus, non-boundary targets in the panoramic image to be detected can be determined based on the panoramic image features. Specifically, non-boundary targets in the panoramic image to be detected can be identified by whether the coordinates within the target detection candidate region include the coordinates of the boundary position of the panoramic image to be detected. When the coordinates within the target detection candidate region do not include the coordinates of the boundary position of the panoramic image to be detected, that is, when all coordinates within the target detection candidate region are within the boundary range of the panoramic image to be detected, the target detected in the target detection candidate region is considered a non-boundary target, and the corresponding non-cyclic bounding box is obtained. In this embodiment, by extracting the panoramic image features of the target detection candidate region, the non-cyclic bounding boxes corresponding to non-boundary targets can be effectively detected, ensuring the target detection effect.

[0080] In one embodiment, such as Figure 6 As shown, step 405 includes:

[0081] Step 601: Extract panoramic image features of target detection candidate regions using a preset target deformation adaptive convolution operator.

[0082] Step 603: Based on the panoramic image features, determine the boundary position target in the panoramic image to be detected, and obtain the detection target.

[0083] Step 605: Identify the target attributes between the first detection target and the second detection target based on panoramic image features. The first detection target and the second detection target are detection targets in relative positions in the panoramic image.

[0084] Step 607: When the target attribute indicates that the first detection target and the second detection target are the same detection target, obtain the circular bounding boxes corresponding to the first detection target and the second detection target.

[0085] Panoramic image features refer to the features of targets located at the boundaries of a panoramic image. By using a pre-defined target deformation adaptive convolution operator, the features can be effectively adapted to target deformation, enabling feature extraction from the panoramic image and thus effectively extracting the corresponding boundary features from the target detection candidate region. Detected targets are those located at the boundaries of the panoramic image, segmented into two sides by the panoramic image. Target attributes are specifically used to determine whether two relatively positioned detected targets (first and second targets) are the same target. When two relatively positioned detected targets are the same target, their target attributes are the same. When two relatively positioned detected targets are not the same target, their target attributes are different.

[0086] Specifically, when identifying the coordinates at the boundary positions, the target may have already been segmented into two opposing boundaries in the panoramic image, resulting in target deformation. Therefore, a pre-defined target deformation adaptive convolution operator can be used to extract the panoramic image features corresponding to these targets. Based on the extracted features, it is further determined which targets in the target detection candidate region belong to the detection targets, and the target attributes corresponding to two detection targets in relative positions are identified. For example, for a panoramic image to be detected with a width of latitude 0-2π and a height of longitude 0-π, a two-dimensional plane coordinate system can be established with the lower left endpoint of the image as the origin, the width direction of the image as the X-axis, and the height direction of the image as the Y-axis. Then, the boundary positions in the panoramic image to be detected are the left boundary at X=0 and the right boundary at X=2π. For detection targets in relative positions, this specifically refers to detection targets containing the same Y-axis coordinates. For example, if the coordinates of a detection target A are identified as (0, 0.5π), then the detection target B containing the coordinates (2π, 0.5π) can be determined as the boundary position target of the relative position of detection target A. Then, based on the panoramic image features extracted by the convolutional neural network, further identification and judgment can be made to determine whether two detected targets in relative positions are the same. When the target attributes indicate that the detected targets in relative positions are the same, the corresponding cyclic bounding box can be obtained. Specifically, corresponding bounding boxes can be added outside the two detected targets to form the cyclic bounding boxes corresponding to the detected targets. In this embodiment, by extracting the panoramic image features of the target detection candidate region, the cyclic bounding boxes corresponding to the detected targets can be effectively detected, ensuring the target detection effect.

[0087] In one embodiment, such as Figure 7 As shown, step 407 includes:

[0088] Step 702: Obtain the confidence level corresponding to the bounding box.

[0089] Step 704: Sort the bounding boxes according to their confidence scores to obtain a list of bounding boxes.

[0090] Step 706: Add the bounding box with the highest confidence to the object detection result list, delete the bounding box with the highest confidence from the bounding box list, and update the bounding box list.

[0091] Step 708: Obtain the interaction ratio between the bounding box with the highest confidence and the bounding boxes in the updated bounding box list.

[0092] Step 710: Delete bounding boxes from the bounding box list whose interaction ratio is greater than the preset interaction ratio threshold.

[0093] Step 712: Determine if the bounding box list is empty. If the bounding box list is not empty, return to step 706. If the updated bounding box list is empty, proceed to step 714 and output the latest object detection result list.

[0094] Before step 708, the method further includes: modifying the calculation rules for the interaction ratio between bounding boxes to make the calculation rules applicable to cyclic bounding boxes.

[0095] In this embodiment, the application specifically uses non-maximum suppression (NMS) to filter the bounding boxes obtained during target detection, eliminating duplicate bounding boxes to improve the accuracy of target detection. NMS, also known as the non-maximum suppression algorithm, searches for local maxima and suppresses maxima. Confidence, also called reliability, confidence level, or confidence coefficient, refers to the probability that the estimated value is within a certain allowable error range compared to the population parameter. Therefore, a probabilistic approach, such as interval estimation in mathematical statistics, is used. This probability is called the confidence level. In this embodiment, for a bounding box, the convolutional neural network calculates the predicted probability that the detected target within the bounding box belongs to each preset category, and selects the largest predicted probability as P, which is called the confidence level of the bounding box. Intersection over Union (IOU) is a concept used in object detection. It is the overlap ratio between the generated candidate bounding boxes and the original ground truth bounding boxes, i.e., the ratio of their intersection to their union.

[0096] Specifically, in the scheme of this application, after obtaining the bounding boxes corresponding to the candidate regions for target detection, since duplicate bounding boxes will be stacked in the same position, a non-maximum suppression algorithm can be used to filter the bounding boxes obtained in the target detection process to exclude duplicate bounding boxes. During non-maximum suppression, the extracted bounding boxes are sorted according to their confidence levels to obtain a corresponding bounding box list. Then, the bounding box with the highest confidence level is selected from the bounding box list as the subsequent judgment benchmark, and all predicted bounding boxes whose intersection-union ratio (IU) with the benchmark bounding box is greater than a preset interaction ratio threshold are removed from the list. This process is repeated multiple times until there are no bounding boxes left in the bounding box list, at which point the loop ends, and all bounding boxes in the target detection result list at this point are the final target detection results. Simultaneously, this embodiment can also modify the calculation rules for the interaction ratio between bounding boxes so that the coordinates outside the image range in the cyclic bounding box are not truncated (e.g., negative coordinates) to make the calculation rules applicable to cyclic bounding boxes. In this embodiment, repeatedly filtering bounding boxes by confidence level can effectively remove other redundant bounding boxes that overlap with the target detection results, improving the efficiency of target detection.

[0097] In one embodiment, before step 502, the method further includes: identifying the position of the cyclic bounding box in the bounding box; and mapping the cyclic bounding box of any boundary in the panoramic image to be detected to the boundary corresponding to the boundary where the cyclic bounding box is located, based on the position of the cyclic bounding box.

[0098] Specifically, in this application, since the cyclic bounding boxes corresponding to the targets at the boundaries of the panoramic image are identified, for the same target at the boundary, there is a corresponding cyclic bounding box on both the left and right sides. Since the target at the edges of both sides is the same, the cyclic bounding box of any boundary in the panoramic image to be detected can be mapped to the corresponding boundary of the cyclic bounding box based on its position. That is, the cyclic bounding boxes on the left (right) side are mapped to the right (left) side. At this time, the repeated rectangles of the same target will be stacked in the same position, thus facilitating the subsequent interaction ratio calculation process. In this embodiment, by mapping the cyclic bounding box of any boundary in the panoramic image to be detected to the corresponding boundary of the cyclic bounding box, the processing efficiency of the non-maximum suppression process is improved, further reducing the repetition of targets within the panoramic image to be detected, and ensuring the detection efficiency of the target.

[0099] In one embodiment, extracting the target detection candidate region of the panoramic image to be detected includes:

[0100] The target detection candidate region of the panoramic image to be detected is extracted by selecting a search algorithm.

[0101] Specifically, selection search is used to obtain candidate regions for object detection. The main idea of ​​the selection search algorithm is that the regions in the image to be detected where objects may exist should have some similarity or continuity. Therefore, selection search uses a sub-region merging method to extract candidate regions for object detection based on this idea. First, the input image is segmented to generate many small sub-regions. Second, regions are merged based on the similarity between these sub-regions (similarity criteria mainly include color, texture, size, etc.), and the merging of regions is continuously iterated. In each iteration, the circumscribed rectangles of these merged sub-regions are used, and these circumscribed rectangles are the candidate regions for object detection. In one embodiment, the specific process of selection search includes: step 0: Generate a region set R. step 1: Calculate the similarity S = {s1, s2, ...} of each adjacent region in the region set R; step 2: Find the two regions with the highest similarity, merge them into a new set, and add it to R; step 3: Remove all subsets related to step 2 from S; step 4: Calculate the similarity between the new set and all subsets; step 5: Jump to step 2 until S is empty. In this embodiment, by selecting a search method, candidate regions for target detection can be effectively extracted from the panoramic image to be detected.

[0102] It should be understood that, although Figure 4-7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 4-7 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0103] In one embodiment, such as Figure 8 As shown, a target detection device for panoramic images is provided, comprising:

[0104] The data acquisition module 801 is used to acquire the panoramic image to be detected.

[0105] The region extraction module 803 is used to extract candidate regions for target detection in the panoramic image to be detected.

[0106] The convolution processing module 805 is used to perform convolution processing on the target detection candidate region through a convolutional neural network containing a preset target deformation adaptive convolution operator, and obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent the bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0107] The target detection module 807 is used to filter the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

[0108] In one embodiment, the convolution processing module 805 is specifically used to: extract panoramic image features of the target detection candidate region by using a preset target deformation adaptive convolution operator; determine the non-boundary position target in the panoramic image to be detected based on the panoramic image features, and obtain the non-cyclic bounding box corresponding to the non-boundary position target.

[0109] In one embodiment, the convolution processing module 805 is specifically used to: determine the boundary position target in the panoramic image to be detected based on the panoramic image features, and obtain the detection target; identify the target attributes between the first detection target and the second detection target based on the panoramic image features, wherein the first detection target and the second detection target are detection targets in relative positions; when the target attributes indicate that the first detection target and the second detection target are the same detection target, obtain the cyclic bounding box corresponding to the first detection target and the second detection target.

[0110] In one embodiment, the target detection module 809 is specifically used for: obtaining the confidence scores corresponding to the bounding boxes; sorting the bounding boxes according to the confidence scores to obtain a bounding box list; adding the bounding box with the highest confidence score to the target detection result list, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; obtaining the interaction ratio between the bounding box with the highest confidence score and the bounding boxes in the updated bounding box list; deleting the bounding boxes in the bounding box list whose interaction ratio is greater than a preset interaction ratio threshold; returning to the steps of selecting the bounding box with the highest confidence score as the target detection result, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; and outputting the latest target detection result list when the updated bounding box list is empty.

[0111] In one embodiment, the target detection module 809 is further configured to: identify the position of the cyclic bounding box in the bounding box; and map the cyclic bounding box of any boundary in the panoramic image to be detected to the boundary corresponding to the boundary where the cyclic bounding box is located, based on the position of the cyclic bounding box.

[0112] In one embodiment, the target detection module 809 is further configured to: modify the calculation rules for the interaction ratio between bounding boxes so that the calculation rules are applicable to cyclic bounding boxes.

[0113] In one embodiment, the region extraction module 803 is specifically used to: extract candidate regions for target detection of the panoramic image to be detected by selecting a search algorithm.

[0114] Specific limitations regarding the target detection device for panoramic images can be found in the limitations of the target detection method for panoramic images described above, and will not be repeated here. Each module in the aforementioned target detection device for panoramic images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0115] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores traffic forwarding data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a target detection method for panoramic images.

[0116] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0118] Acquire the panoramic image to be detected;

[0119] Extract candidate regions for target detection from the panoramic image to be detected;

[0120] The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent the bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0121] By filtering the bounding boxes using a preset overlay border filtering algorithm, the target detection results corresponding to the panoramic image to be detected are obtained.

[0122] In one embodiment, when the processor executes the computer program, it further performs the following steps: extracting panoramic image features of the target detection candidate region through a convolutional neural network; determining the non-boundary target in the panoramic image to be detected based on the panoramic image features; and obtaining the non-cyclic bounding box corresponding to the non-boundary target.

[0123] In one embodiment, when the processor executes the computer program, it further implements the following steps: determining the boundary position target in the panoramic image to be detected based on the panoramic image features, and obtaining the detection target; identifying the target attributes between the first detection target and the second detection target based on the panoramic image features, wherein the first detection target and the second detection target are detection targets in relative positions; and obtaining the cyclic bounding boxes corresponding to the first detection target and the second detection target when the target attributes indicate that the first detection target and the second detection target are the same detection target.

[0124] In one embodiment, when the processor executes the computer program, it further implements the following steps: obtaining the confidence scores corresponding to the bounding boxes; sorting the bounding boxes according to the confidence scores to obtain a bounding box list; adding the bounding box with the highest confidence score to the target detection result list, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; obtaining the interaction ratio between the bounding box with the highest confidence score and the bounding boxes in the updated bounding box list; deleting the bounding boxes in the bounding box list whose interaction ratio is greater than a preset interaction ratio threshold; returning to the steps of selecting the bounding box with the highest confidence score as the target detection result, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; and outputting the latest target detection result list when the updated bounding box list is empty.

[0125] In one embodiment, when the processor executes the computer program, it further performs the following steps: identifying the position of the loop bounding box in the bounding box; and mapping the loop bounding box of any boundary in the panoramic image to be detected to the boundary corresponding to the boundary where the loop bounding box is located, based on the position of the loop bounding box.

[0126] In one embodiment, when the processor executes the computer program, it further performs the following steps: modifying the calculation rules for the interaction ratio between bounding boxes so that the calculation rules apply to cyclic bounding boxes.

[0127] In one embodiment, the processor, when executing the computer program, also performs the following steps: extracting candidate regions for target detection from the panoramic image to be detected by selecting a search algorithm.

[0128] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0129] Acquire the panoramic image to be detected;

[0130] Extract candidate regions for target detection from the panoramic image to be detected;

[0131] The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent the bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected.

[0132] By filtering the bounding boxes using a preset overlay border filtering algorithm, the target detection results corresponding to the panoramic image to be detected are obtained.

[0133] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting panoramic image features of the target detection candidate region through a convolutional neural network; determining the non-boundary target in the panoramic image to be detected based on the panoramic image features; and obtaining the non-cyclic bounding box corresponding to the non-boundary target.

[0134] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the boundary position target in the panoramic image to be detected based on the panoramic image features, thereby obtaining the detection target; identifying the target attributes between the first detection target and the second detection target based on the panoramic image features, wherein the first detection target and the second detection target are detection targets in relative positions; and obtaining the cyclic bounding boxes corresponding to the first detection target and the second detection target when the target attributes indicate that the first detection target and the second detection target are the same detection target.

[0135] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: obtaining the confidence scores corresponding to the bounding boxes; sorting the bounding boxes according to the confidence scores to obtain a bounding box list; adding the bounding box with the highest confidence score to the target detection result list, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; obtaining the interaction ratio between the bounding box with the highest confidence score and the bounding boxes in the updated bounding box list; deleting the bounding boxes in the bounding box list whose interaction ratio is greater than a preset interaction ratio threshold; returning to the steps of selecting the bounding box with the highest confidence score as the target detection result, deleting the bounding box with the highest confidence score from the bounding box list, and updating the bounding box list; and outputting the latest target detection result list when the updated bounding box list is empty.

[0136] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: identifying the position of the cyclic bounding box in the bounding box; and mapping the cyclic bounding box of any boundary in the panoramic image to be detected to the boundary corresponding to the boundary where the cyclic bounding box is located, based on the position of the cyclic bounding box.

[0137] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: modifying the calculation rules for the interaction ratio between bounding boxes to make the calculation rules applicable to cyclic bounding boxes. In another embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting candidate regions for object detection from the panoramic image to be detected by selecting a search algorithm.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A target detection method for panoramic images, the method comprising: Acquire the panoramic image to be detected; Extract the target detection candidate region from the panoramic image to be detected; The target detection candidate region is convolved by a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected. The convolutional neural network is used to output the corresponding cyclic bounding box for the same target in the boundary part of the panoramic image to be detected. The bounding box is filtered by a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

2. The method according to claim 1, characterized in that, The bounding box further includes a non-cyclic bounding box. The non-cyclic bounding box corresponding to the target detection candidate region is obtained by performing convolution processing on the target detection candidate region using a convolutional neural network containing a preset target deformation adaptive convolution operator. The panoramic image features of the target detection candidate region are extracted using the convolutional neural network. Based on the panoramic image features, non-boundary targets in the panoramic image to be detected are determined, and non-cyclic bounding boxes corresponding to the non-boundary targets are obtained.

3. The method according to claim 1, characterized in that, The target detection candidate region is convolved using a convolutional neural network that includes a preset target deformation adaptive convolution operator to obtain the corresponding cyclic bounding box of the target detection candidate region, including: Panoramic image features of the target detection candidate region are extracted using a preset target deformation adaptive convolution operator; Based on the features of the panoramic image, the target at the boundary position in the panoramic image to be detected is determined, and the detected target is obtained; Based on the panoramic image features, target attributes between the first detection target and the second detection target are identified, where the first detection target and the second detection target are detection targets in relative positions in the panoramic image; When the target attribute indicates that the first detection target and the second detection target are the same detection target, the corresponding circular bounding boxes of the first detection target and the second detection target are obtained.

4. The method according to claim 1, characterized in that, The step of filtering the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected includes: Obtain the confidence level corresponding to the bounding box; The bounding boxes are sorted according to the confidence level to obtain a bounding box list; Add the bounding box with the highest confidence to the target detection result list, remove the bounding box with the highest confidence from the bounding box list, and update the bounding box list; Obtain the interaction ratio between the bounding box with the highest confidence level and the bounding boxes in the updated bounding box list; Delete bounding boxes from the bounding box list whose interaction ratio is greater than a preset interaction ratio threshold. The steps are: return to select the bounding box with the highest confidence as the target detection result, delete the bounding box with the highest confidence from the bounding box list, and update the bounding box list; When the updated bounding box list is empty, output the latest list of target detection results.

5. The method according to claim 4, characterized in that, Before obtaining the confidence level corresponding to the bounding box, the method further includes: Identify the position of the circular bounding box within the bounding box; Based on the position of the cyclic bounding box, the cyclic bounding box of any boundary in the panoramic image to be detected is mapped to the boundary corresponding to the boundary where the cyclic bounding box is located.

6. The method according to claim 4, characterized in that, Before obtaining the interaction ratio between the bounding box with the highest confidence score and the bounding boxes in the updated bounding box list, the method further includes: Modify the calculation rules for the interaction ratio between bounding boxes to make the calculation rules applicable to cyclic bounding boxes.

7. The method according to claim 1, characterized in that, The extraction of the target detection candidate region from the panoramic image to be detected includes: The target detection candidate region of the panoramic image to be detected is extracted by selecting a search algorithm.

8. A target detection device for panoramic images, characterized in that, The device includes: The data acquisition module is used to acquire the panoramic image to be detected; The region extraction module is used to extract the target detection candidate region of the panoramic image to be detected; The convolution processing module is used to perform convolution processing on the target detection candidate region through a convolutional neural network containing a preset target deformation adaptive convolution operator to obtain the bounding box corresponding to the target detection candidate region. The bounding box includes a cyclic bounding box, which is used to represent a bounding box whose coordinate range exceeds the boundary of the panoramic image to be detected. The convolutional neural network is used to output the corresponding cyclic bounding box for the same target in the boundary part of the panoramic image to be detected. The target detection module is used to filter the bounding box using a preset overlay border filtering algorithm to obtain the target detection result corresponding to the panoramic image to be detected.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images