Target recognition method, device and equipment based on panoramic stitched image

Abstract

The application provides a target recognition method, device and equipment based on a panoramic spliced image, relates to the technical field of computer vision and image processing, and the method comprises the following steps: dividing a to-be-recognized image according to a preset region grid to obtain a to-be-recognized image region; using a block recognition model to recognize the to-be-recognized image region, obtaining all recognition results, expanding each recognition result outward to obtain all current target regions, and determining all to-be-recognized target regions according to all recognized regions and all current target regions; determining a target batch recognition model according to the number of all to-be-recognized target regions, using the target batch recognition model to recognize all to-be-recognized target regions, obtaining all target recognition objects, and displaying all target recognition objects in the to-be-recognized image. The block recognition model is used to recognize a single divided region, and the batch recognition model is used to recognize the extracted target regions in batches, so that the calculation amount of each recognition is reduced.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a target recognition method, apparatus and device based on panoramic stitched images. Background Technology

[0002] In the field of image recognition, small target recognition (such as small object detection in drone surveillance and small lesion recognition in medical images) has always been a challenge. Small targets typically occupy a small area in an image and often blend into the background, resulting in low accuracy of general target detection methods when recognizing small targets. Deep learning models have significantly improved the performance of small target detection through methods such as multi-scale feature fusion and feature pyramid networks. However, in high-resolution panoramic images, balancing the accuracy and speed of small target detection remains a challenge that urgently needs to be addressed.

[0003] The Single Shot MultiBox Detector (SSD) is an efficient object detection method that fuses feature maps at different scales to effectively detect objects of varying sizes. Its ability to predict at different resolutions allows for more detailed anchor box design and feature adjustment. The YOLO series of real-time object detection algorithms, starting with YOLO v5, can be configured with a P2 detection layer to enhance the detection of small objects. The Slice-Assisted Hyper-Inference Library (SAHI) is primarily used to improve detection accuracy when objects in datasets are small. SAHI can be applied to various existing object detection networks, and is particularly suitable for scenarios such as drone aerial imagery. While existing object detection methods, such as SSD and the YOLO series, demonstrate competitiveness in terms of accuracy and speed, they still have limitations when processing high-resolution panoramic stitched images. For devices with limited computing power, performing real-time inference on multiple high-resolution images to meet the needs of small object recognition remains a challenge.

[0004] Therefore, a more efficient and accurate method is needed for the identification of small targets in high-resolution panoramic stitched images. However, there is currently no technical solution that can solve the above-mentioned technical problems, and there is no target recognition method, device or equipment based on panoramic stitched images. Summary of the Invention

[0005] This invention provides a target recognition method, apparatus, and device based on panoramic stitched images, aiming to solve the problem of small target recognition in high-resolution panoramic stitched images.

[0006] In a first aspect, the present invention provides a target recognition method based on panoramic stitched images, comprising:

[0007] For an image to be identified acquired at any given time, the image to be identified is divided into blocks according to a preset region grid to obtain the image region to be identified corresponding to that time.

[0008] Using a block recognition model, the image region to be recognized is identified, and all recognition results output by the block recognition model are obtained. Each recognition result is expanded to obtain all current target regions corresponding to the time. Based on all recognized regions corresponding to all preset times before the time and all current target regions, all target regions to be recognized are determined.

[0009] A target batch recognition model is determined based on the number of all target regions to be recognized. The target batch recognition model is used to recognize all target regions to be recognized, and all target recognition objects output by the target batch recognition model are obtained. All target recognition objects are displayed in the image to be recognized.

[0010] Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

[0011] According to the target recognition method based on panoramic stitched images provided by the present invention, before dividing the image to be recognized into blocks according to a preset region grid, the method further includes:

[0012] Use a panoramic pod to acquire camera images from different angles;

[0013] All camera images are stitched together to obtain a panoramic stitched image. Redundancy in the overlapping parts of the panoramic stitched image is removed to obtain the image to be identified.

[0014] According to the target recognition method based on panoramic stitched images provided by the present invention, the preset area grid has a total of 15 blocks, wherein the cameras in different directions include a central camera, a left camera, a right camera, an upper camera, and a lower camera, and the camera image acquired by each camera is divided into 3 blocks;

[0015] The step of dividing the image to be identified into blocks according to a preset region grid to obtain the image region to be identified at the specified time includes:

[0016] The image to be identified is divided into blocks according to a preset region grid, and the block region corresponding to the block at the given time is obtained.

[0017] The segmented region is determined as the image region to be identified at the time specified in the image to be identified.

[0018] According to the target recognition method based on panoramic stitched images provided by the present invention, before using a block recognition model to recognize the region of the image to be recognized and obtaining all recognition results output by the block recognition model, the method further includes:

[0019] The first initial recognition model is trained using the original sample images corresponding to the original resolution and the sample recognition results corresponding to each original sample image to obtain the block recognition model;

[0020] Both the image to be identified and the region of the image to be identified are at their original resolution.

[0021] According to the target recognition method based on panoramic stitched images provided by the present invention, before recognizing all the target regions to be recognized using the target batch recognition model, the method further includes:

[0022] The second initial recognition model is trained using sample recognition images corresponding to a preset resolution and the sample recognition object corresponding to each sample recognition image to obtain the target batch recognition model;

[0023] The preset resolution is smaller than the original resolution.

[0024] According to the target recognition method based on panoramic stitched images provided by the present invention, the step of expanding each recognition result to obtain all current target regions corresponding to the given time includes:

[0025] Determine the center point of the recognition result, and make the center point of the matrix region of a preset size coincide with the center point of the recognition result;

[0026] The matrix region of the preset size is determined as the current target region corresponding to the recognition result;

[0027] By iterating through all recognition results, all current target regions corresponding to the given time are obtained.

[0028] According to the target recognition method based on panoramic stitched images provided by the present invention, the step of determining the target batch recognition model based on the number of all target regions to be recognized includes:

[0029] If the number of all target regions to be identified is less than or equal to 8, the target batch identification model is determined to be an 8-batch identification model.

[0030] If the number of all target regions to be identified is greater than 8 and less than or equal to 16, the target batch identification model is determined to be a 16-batch identification model.

[0031] If the number of all target regions to be identified is greater than 16 and less than or equal to 32, the target batch identification model is determined to be a 32-batch identification model.

[0032] If the number of all target regions to be identified is greater than 32 and less than or equal to 64, the target batch identification model is determined to be a 64-batch identification model.

[0033] If the number of all target regions to be identified is greater than 64, the target batch identification model is determined to be a 128-batch identification model.

[0034] According to the target recognition method based on panoramic stitched images provided by the present invention, the target object to be recognized is a vehicle. Before displaying all target objects to be recognized in the image to be recognized, the method further includes:

[0035] For any target region to be identified, if more than one target object is identified, all target objects in the target region to be identified are filtered out, and only one target object is retained.

[0036] Secondly, a target recognition device based on panoramic stitched images is provided, comprising:

[0037] The segmentation unit is used to segment the image to be identified according to a preset region grid for an image to be identified acquired at any time, so as to obtain the image region to be identified corresponding to the time in the image to be identified;

[0038] The recognition unit is used to recognize the image region to be recognized using a block recognition model, obtain all recognition results output by the block recognition model, expand each recognition result to obtain all current target regions corresponding to the time, and determine all target regions to be recognized based on all recognized regions corresponding to all preset times before the time and all current target regions.

[0039] A determining unit is configured to determine a target batch recognition model based on the number of all target regions to be identified, use the target batch recognition model to identify all target regions to be identified, obtain all target recognition objects output by the target batch recognition model, and display all target recognition objects in the image to be identified.

[0040] Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

[0041] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target recognition method based on panoramic stitched images.

[0042] This invention divides the image to be identified into blocks according to a preset regional grid, and uses a block recognition model to identify individual blocks. Simultaneously, a batch recognition model is used to identify the extracted target regions in batches, significantly reducing the computational load for each identification. This method effectively shortens the identification time and improves the detection speed. Dynamically selecting a suitable batch recognition model based on the number of target regions to be identified further optimizes the identification process and improves overall identification efficiency. Both the image to be identified and the image regions to be identified maintain their original resolution, ensuring the accuracy of the identification results. By optimizing the identification process and model selection, the identification time is significantly shortened. While maintaining a certain level of detection accuracy, it also has a high detection speed, meeting the requirements for real-time / near real-time image target recognition on embedded devices. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is one of the flowcharts illustrating the target recognition method based on panoramic stitched images provided by the present invention;

[0045] Figure 2 This is the second flowchart of the target recognition method based on panoramic stitched images provided by the present invention;

[0046] Figure 3 This is a schematic diagram of the image area directly below the panoramic stitched image provided by the present invention;

[0047] Figure 4This is a schematic diagram of the side image region based on panoramic stitched images provided by the present invention;

[0048] Figure 5 This is a schematic diagram of dividing the image region directly below into blocks, as provided by the present invention;

[0049] Figure 6 This is a schematic diagram of dividing the side image region into blocks according to the present invention;

[0050] Figure 7 This is a schematic diagram of the target recognition device based on panoramic stitched images provided by the present invention;

[0051] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] This invention provides an accelerated method for small target recognition in panoramic stitched images at their original resolution. This method can be widely applied to remote sensing image analysis, ecological environment monitoring, industrial production line and object inspection, military and defense industries, etc. For example, a certain type of panoramic agile pod uses a panoramic image stitched together from five 2880*1860 pixel images taken by five 5-megapixel cameras. After aligning the camera video frames, each recognition requires processing five images. If the dimensions of the five images are downsampled by a factor of three to 960*620 pixels before recognition, the accuracy of small target recognition will be reduced. If downsampling is not performed and recognition is performed at the original resolution, the computational load for target recognition is approximately 65 times that of recognizing a single 640*640 pixel image, which is insufficient for low-latency applications.

[0054] Based on this technical problem, the present invention provides a target recognition method based on panoramic stitched images. Figure 1 This is one of the flowcharts illustrating the target recognition method based on panoramic stitched images provided by the present invention. The target recognition method based on panoramic stitched images includes:

[0055] Step 101: For an image to be identified acquired at any time, the image to be identified is divided into blocks according to a preset region grid to obtain the image region to be identified corresponding to the time in the image to be identified;

[0056] Step 102: Using a block recognition model, identify the image region to be identified, obtain all recognition results output by the block recognition model, expand each recognition result to obtain all current target regions corresponding to the time, and determine all target regions to be identified based on all identified regions corresponding to all preset times before the time and all current target regions.

[0057] Step 103: Determine the target batch recognition model based on the number of all target regions to be recognized, use the target batch recognition model to recognize all target regions to be recognized, obtain all target recognition objects output by the target batch recognition model, and display all target recognition objects in the image to be recognized;

[0058] Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

[0059] In step 101, before dividing the image to be identified into blocks according to a preset region grid, the method further includes:

[0060] Use a panoramic pod to acquire camera images from different angles;

[0061] All camera images are stitched together to obtain a panoramic stitched image. Redundancy in the overlapping parts of the panoramic stitched image is removed to obtain the image to be identified.

[0062] Optionally, the panoramic pod is deployed in a suitable location to ensure that a panoramic view of the desired area can be captured. Multiple cameras within the panoramic pod (potentially including center, left, right, top, and bottom cameras) are triggered simultaneously or sequentially to capture images from different orientations. Each camera captures an image from its own viewpoint and transmits these images to the processing system. The processing system first registers all the acquired camera images, i.e., determines their relative positions and orientations for correct stitching. A preset image stitching algorithm is used to seamlessly stitch all the registered images into a single panoramic image. In the panoramic image, redundant areas caused by the overlap of different camera images are detected. Based on the detection results of the overlapping areas, these redundant parts are selectively removed to obtain a clear, non-overlapping image to be identified.

[0063] Figure 3This is a schematic diagram of the image area directly below a panoramic stitched image provided by the present invention. Figure 4 This is a schematic diagram of the side image area based on panoramic stitched images provided by the present invention. After the pod is installed on the drone, the central camera is vertically downward, and the four side cameras are tilted forward, backward, left, and right at 45 degrees respectively. All five cameras have a resolution of 2880*1860 pixels, and the stitched panoramic image is as follows. Figure 3 As shown, the areas with yellow dots correspond to the images captured by the central camera. For the stitched panoramic image, only the central square area is usable. The other areas overlap with the images captured by the side cameras, so these areas can be cropped out, leaving only the central 1728*1728 pixel area.

[0064] like Figure 4 As shown, the area with yellow dots corresponds to the image taken by the side camera. The upper part shows a very far distance. According to experimental statistics, the size of the vehicle target is generally less than 16*16 pixels, and the tilt angle is large, making it very likely to be occluded, which makes it difficult to identify under the current conditions. Considering that the actual needs are more concerned with the vehicle target at close range, about one-third of the upper part can be discarded, and only the lower 2688*1120 pixel area can be retained.

[0065] Optionally, the preset area grid has 15 blocks, wherein the cameras in different positions include a central camera, a left camera, a right camera, an upper camera, and a lower camera, and the camera image acquired by each camera is divided into 3 blocks;

[0066] The step of dividing the image to be identified into blocks according to a preset region grid to obtain the image region to be identified at the specified time includes:

[0067] The image to be identified is divided into blocks according to a preset region grid, and the block region corresponding to the block at the given time is obtained.

[0068] The segmented region is determined as the image region to be identified at the time specified in the image to be identified.

[0069] Optionally, the present invention first defines a preset region grid containing 15 blocks. This grid is designed to cover the entire image to be identified, ensuring that for each frame of the image, it is processed according to this preset region grid, and each part can be effectively processed, thereby obtaining the block region corresponding to the frame image at each time. Considering that the panoramic image is stitched together from images taken from five cameras in different directions—the center camera, the left camera, the right camera, the top camera, and the bottom camera—and each camera image is evenly divided into three blocks, the entire panoramic image is divided into 15 blocks, matching a preset region grid. At any given moment, based on the preset region grid, the image acquired at that moment is selected for block processing, thereby obtaining the block region image corresponding to that moment. Those skilled in the art understand that since this invention relates to dynamic image processing, a picture to be identified is obtained at each moment. The image to be identified at that moment is then processed using the preset region grid. Since each moment corresponds to processing one block in the preset region grid, the sum of the number of moments of all preset moments and the current moment is the same as the number of blocks in the preset region grid. Therefore, the corresponding block requiring identification processing can be determined from the image to be identified at that moment. This block is the "image region to be identified corresponding to the moment in the image to be identified."

[0070] This invention divides a panoramic image into 15 blocks and uses a block recognition model to process one of the blocks. This method significantly reduces the computational complexity of a single processing step and improves the overall processing efficiency. It processes only one block at a time, allowing the algorithm to focus more on the local details of the image, thereby improving the accuracy of target recognition. Since it processes only a small area at a time, this method effectively utilizes computing resources and avoids unnecessary waste, making it particularly suitable for resource-constrained environments.

[0071] In step 102, the present invention uses a pre-trained block recognition model to perform target recognition on the current image region to be recognized, collects all recognition results output by the model, and expands each recognition result outward. This expansion can be based on the boundary of the recognized target object to obtain the current target region. Then, it combines the recognized regions from all preset time points. These recognized regions are obtained through the same process, that is, by using the block recognition model to recognize and expand the images to be recognized corresponding to a preset number of time points before this moment, to determine all target regions to be recognized. Those skilled in the art will understand that, for example, if there are 15 blocks, then 15 time points can be set, where each time point corresponds to the image to be recognized corresponding to a specified block in the preset region grid. At this time, cyclic recognition can be performed. For example, after all the recognition results at the current time point are recognized, all the historical recognition results of the previous 14 time points before the current time point are determined. Then, all the historical recognition results corresponding to the previous 14 time points are expanded outward to obtain all the historical target regions corresponding to the previous 14 time points, which are then determined as all the recognized regions. Finally, all the recognized regions are added to all the current target regions corresponding to the current time point to form all the target regions to be recognized.

[0072] Optionally, before using a block recognition model to identify the image region to be identified and obtaining all recognition results output by the block recognition model, the method further includes:

[0073] The first initial recognition model is trained using the original sample images corresponding to the original resolution and the sample recognition results corresponding to each original sample image to obtain the block recognition model;

[0074] Both the image to be identified and the region of the image to be identified are at their original resolution.

[0075] Optionally, this invention collects sample images at their original resolution. These images are used to train a block recognition model. For each original sample image, corresponding sample recognition results are labeled or generated. These results can include object categories, location information, etc., depending on the requirements of the recognition task. A suitable first initial recognition model is selected; this model can be any deep learning model suitable for image recognition tasks, such as a convolutional neural network. The first initial recognition model is trained using the original resolution sample images and the corresponding sample recognition results. During training, the model learns to extract useful features from the original resolution images and performs recognition based on these features. Through multiple iterations and optimizations, a trained block recognition model is obtained. The trained block recognition model is then used to recognize the image region to be recognized. The model extracts features from the region and outputs recognition results based on these features.

[0076] Optionally, before using the target batch recognition model to identify all the target regions to be identified, the method further includes:

[0077] The second initial recognition model is trained using sample recognition images corresponding to a preset resolution and the sample recognition object corresponding to each sample recognition image to obtain the target batch recognition model;

[0078] The preset resolution is smaller than the original resolution.

[0079] Optionally, sample recognition images with a preset resolution are collected. These images have a lower resolution than the original resolution, usually to reduce computational complexity and improve processing speed. For each sample recognition image, the corresponding sample recognition object is determined, such as the target vehicle. A suitable second initial recognition model is selected as the training starting point. This model can be a convolutional neural network in deep learning or other models suitable for image recognition. The second initial recognition model is trained using sample recognition images with the preset resolution and the corresponding sample recognition object. When there is a batch of target regions to be recognized, the trained target batch recognition model is used for recognition.

[0080] Optionally, the step of expanding each recognition result to obtain all current target regions corresponding to the time point includes:

[0081] Determine the center point of the recognition result, and make the center point of the matrix region of a preset size coincide with the center point of the recognition result;

[0082] The matrix region of the preset size is determined as the current target region corresponding to the recognition result;

[0083] By iterating through all recognition results, all current target regions corresponding to the given time are obtained.

[0084] Figure 5 This is a schematic diagram illustrating the segmentation of the image region directly below, as provided by the present invention. Figure 6 This is a schematic diagram of segmenting the side image region according to the present invention. Figure 5 as well as Figure 6 In this context, redundant areas refer to regions in the image captured by the camera that do not require recognition, such as... Figure 5 as well as Figure 6 The gray area is shown; the segmented area is the area to be identified after removing redundant areas, and it is then segmented into blocks, as shown below. Figure 5 as well as Figure 6 As shown in the white area, each region has three sub-regions, but their shapes and sizes differ; the target region is a certain range extending beyond the identified target bounding box, such as... Figure 5 as well as Figure 6 The cyan area is shown in the image.

[0085] Optionally, for each recognition result, its center point is first determined. The center point can be obtained by calculating the geometric center of the recognition result area, that is, by calculating the average value of the boundary coordinates of the area. A matrix area of ​​a preset size is defined. The size of this area can be set according to actual needs, which may depend on the size and shape of the target object or the specific requirements of the recognition task. The center point of this matrix area of ​​the preset size is made to coincide with the center point of the recognition result, ensuring that the matrix area expands with the recognition result as the center. Once the center point of the matrix area coincides with the center point of the recognition result, this matrix area is defined as the current target area corresponding to the recognition result.

[0086] In step 103, the present invention selects a suitable batch recognition model based on the number of target regions to be identified, and uses the selected batch recognition model to identify all target regions to obtain all target recognition objects. These recognition objects are then displayed in the original image to be identified. In such an embodiment, the block recognition model is used to identify a single block region at a time, while the batch recognition model is used to identify multiple target regions in batches at a time. The batch recognition model can process multiple target regions at once, improving recognition efficiency. Selecting a model based on the number of target regions can achieve more accurate and efficient recognition.

[0087] Optionally, determining the target batch recognition model based on the number of all target regions to be identified includes:

[0088] If the number of all target regions to be identified is less than or equal to 8, the target batch identification model is determined to be an 8-batch identification model.

[0089] If the number of all target regions to be identified is greater than 8 and less than or equal to 16, the target batch identification model is determined to be a 16-batch identification model.

[0090] If the number of all target regions to be identified is greater than 16 and less than or equal to 32, the target batch identification model is determined to be a 32-batch identification model.

[0091] If the number of all target regions to be identified is greater than 32 and less than or equal to 64, the target batch identification model is determined to be a 64-batch identification model.

[0092] If the number of all target regions to be identified is greater than 64, the target batch identification model is determined to be a 128-batch identification model.

[0093] This invention utilizes computing resources more effectively and avoids resource waste or overload by dynamically selecting an appropriate batch recognition model based on the number of target areas to be identified. It provides multiple batch model options that can be flexibly adjusted according to actual conditions to meet different recognition needs. Selecting a batch model that matches the number of target areas to be identified ensures that the model runs at its best performance, thereby improving recognition accuracy and speed.

[0094] Optionally, the target object to be identified is a vehicle, and before displaying all target objects in the image to be identified, the method further includes:

[0095] For any target region to be identified, if more than one target object is identified, all target objects in the target region to be identified are filtered out, and only one target object is retained.

[0096] Optionally, for any target area to be identified, first check the number of target objects identified in the area. If more than one target object is identified in a target area to be identified, start a filtering mechanism, such as selecting the object with the highest confidence, selecting the object closest to the center of the area, or using more complex decision logic, such as comprehensively considering confidence, location, vehicle speed and other features. Based on the result of the filtering mechanism, only one target object is retained, and other identified objects in the area are removed or ignored.

[0097] To address the challenge of achieving high-accuracy real-time target recognition from multiple high-resolution images on resource-constrained embedded devices, this invention utilizes the relatively mature YOLO deep learning algorithm for method design and experimental verification. First, two target recognition models are trained and optimized through deployment and inference. For each recognition operation, multiple high-resolution images are read and divided into blocks. One block is directly recognized, while the remaining blocks are expanded by a certain size based on the previously recognized bounding box position and size to extract multiple target regions. These target regions are then recognized, and finally, the target recognition results are integrated and their positions are back-calculated onto the original image.

[0098] This invention divides the image to be identified into blocks according to a preset regional grid, and uses a block recognition model to identify individual blocks. Simultaneously, a batch recognition model is used to identify the extracted target regions in batches, significantly reducing the computational load for each identification. This method effectively shortens the identification time and improves the detection speed. Dynamically selecting a suitable batch recognition model based on the number of target regions to be identified further optimizes the identification process and improves overall identification efficiency. Both the image to be identified and the image regions to be identified maintain their original resolution, ensuring the accuracy of the identification results. By optimizing the identification process and model selection, the identification time is significantly shortened. While maintaining a certain level of detection accuracy, it also has a high detection speed, meeting the requirements for real-time / near real-time image target recognition on embedded devices.

[0099] Figure 2 This is the second flowchart illustrating the target recognition method based on panoramic stitched images provided by this invention, as shown below. Figure 2 As shown:

[0100] Step 1: Create training sets for segmented regions and target regions. The long sides of all images in the segmented region training set should be as equal or similar as possible. For example, in the experiments, the long sides of the images were all 1440 pixels. The original image size captured by the camera during acquisition was 2880*1860 pixels. Each image was uniformly cropped into four images, each 1440*930 pixels. The long sides of the images in the target region training set should be as close as possible to the long side of the cropped target region in actual applications. This is because smaller scaling before inference can preserve the feature information of the original image as much as possible, reducing the loss of recognition accuracy. For example, in the experiments, 60% of the images in the segmented region training set were randomly selected. The target region sample images were cropped by expanding the target bounding box by 16 pixels and adding a random position offset. Experimental statistics showed that the length and width of these target region images mostly ranged between 42*121 pixels, and the added random position offset met the application requirements of real-world scenarios.

[0101] Step 2: Train the block recognition model and the batch recognition model. Train the block recognition model at the original image resolution, and train the batch recognition model at a resolution close to the average width and height of the images in the training set for the target region. For example, in the experiment, the block recognition model was trained at a size of 1440*1440 pixels, and the batch recognition model was trained at a size of 64*64 pixels.

[0102] Step 3: Export and deploy the block recognition model and batch recognition model. According to actual application requirements, export the block recognition model as one or more single-batch inference models of different sizes, and export the batch recognition model as multiple inference models of the same size but different batches. For example, export the block recognition model as two single-batch models of 1728*576 and 896*1120 sizes, and export the batch recognition model as five models of batches 8, 16, 32, 64, and 128. A total of seven inference models are exported. Deploy the exported inference models to the target inference platform. For example, deploy the seven inference models exported in the previous step to the Jetson Or in NX embedded platform, loading them all using TensorRT in the program, and preparing the necessary memory and video memory space.

[0103] Step 4: Read N high-resolution images each time, and load the set of images to be recognized into memory. For example, read 5 2880*1860 images from the camera's shared memory and record the memory pointer address.

[0104] Step 5: Crop out redundant areas in the image that do not need to be recognized. This step is not mandatory and should be selected based on actual needs.

[0105] Step 6: Divide the remaining image area to be identified into blocks. For example, the image area to be identified in the central camera is 1728*1728 pixels in size, which is divided into three blocks of 1728*576 pixels. The image area to be identified in the four side cameras is 2688*1120 pixels in size, and each image is also divided into three blocks of 896*1120 pixels.

[0106] Step 7: Directly identify one of the blocks using the block recognition model. If the previous step yielded 15 image regions, identify the first block at time T1, the second block at time T2, the third block at time T3, and so on, until the 15th block is identified at time T15, the first block at time T16, the second block at time T17, the third block at time T18, and so on, repeating the process. Note that the size of the block to be identified must correspond to the size of the input image for the block recognition model.

[0107] Step 8: Extract multiple target regions from the remaining blocks. Using the position and size of the recognition results from the previous time step as a reference, crop multiple target regions on the current image by expanding outwards by a certain size. If no target bounding boxes have been recognized at time T1, target regions cannot be cropped, and this step and the next step are not executed. At time T2, multiple target regions are cropped using the multiple recognition results from the first block of time T1 as a reference. At time T3, multiple target regions are cropped using the multiple recognition results from the first and second blocks of time T2 as a reference. At time T4, multiple target regions are cropped using the multiple recognition results from the first, second, and third blocks of time T3 as a reference. ... At time T15, multiple target regions are cropped using the multiple recognition results from the first to 14 blocks of time T14 as a reference. At time T16, multiple target regions are cropped using the multiple recognition results from the second to 15 blocks of time T15 as a reference. At time T17, multiple target regions are cropped using the multiple recognition results from the first and third to 15 blocks of time T16 as a reference. Repeat this process.

[0108] Step 9: Use a batch recognition model to identify the target regions. For example, if the number of target regions is 8 or less, use an 8-batch batch recognition model; if the number of target regions is 16 or less, use a 16-batch batch recognition model; if the number of target regions is 32 or less, use a 32-batch batch recognition model, and so on. If the number of target regions is greater than 128, first use a 128-batch batch recognition model to identify the first 128, then identify the remaining regions.

[0109] Step 10: Merge the recognition results of all segmented regions, and back-calculate the target position and size in the original image using the recognition results from steps 7 and 9. Regardless of whether the target is identified using a block recognition model or a batch recognition model, the target's position in the segmented region needs to be added to the clipping position of the segmented region in the original image. When the number of identified targets in a target region is greater than one, factors such as the target's maximum movement speed need to be considered, and filtering should be performed based on the target position and size from the previous recognition result, retaining only one target.

[0110] For a certain model of panoramic agile pod, recognition tests were conducted on Jetson Or in NX. Before using this method to identify ground vehicle targets, without downsampling, the inference time for each of the 5 images was approximately 207 milliseconds. After using this method, the inference time for identifying 5 images was approximately 35 milliseconds. Each loop of 15 times allows for one block-based region recognition, therefore the time interval between two adjacent recognitions of the same block is approximately 500 milliseconds. In flight experiments, the accuracy rate was approximately 92.2% when using the original image resolution, and approximately 87.1% after using this method, a decrease of approximately 5.1%, but still significantly higher than the accuracy rate for identifying small targets using downsampling, simplified network structures, or model quantization. The loss in target recognition accuracy due to this method is mainly reflected in the false negative and false positive rates for small targets during target region recognition. However, identifying and finding a target larger than 16*16 pixels near the center of a 64*64 pixel image generally maintains a high accuracy rate. Therefore, this method or its variants have a certain degree of versatility and can be generalized to other types of target recognition or other deep learning tasks.

[0111] Figure 7 This is a schematic diagram of the target recognition device based on panoramic stitched images provided by the present invention. The target recognition device based on panoramic stitched images includes a segmentation unit 1. The segmentation unit 1 is used to segment the image to be recognized according to a preset region grid for an image to be recognized acquired at any time, so as to obtain the image region to be recognized corresponding to the time in the image to be recognized. The working principle of the segmentation unit 1 can be referred to the aforementioned step 101, and will not be repeated here.

[0112] The target recognition device based on panoramic stitched images further includes a recognition unit 2. The recognition unit 2 is used to recognize the image region to be recognized using a block recognition model, obtain all recognition results output by the block recognition model, expand each recognition result to obtain all current target regions corresponding to the time, and determine all target regions to be recognized based on all recognized regions corresponding to all preset times before the time and all current target regions. The working principle of the recognition unit 2 can be referred to the aforementioned step 102, and will not be repeated here.

[0113] The target recognition device based on panoramic stitched images further includes a determination unit 3. The determination unit 3 is used to determine a target batch recognition model based on the number of all target regions to be recognized, use the target batch recognition model to recognize all target regions to be recognized, obtain all target recognition objects output by the target batch recognition model, and display all target recognition objects in the image to be recognized. The working principle of the determination unit 3 can be referred to the aforementioned step 103, and will not be repeated here.

[0114] Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

[0115] This invention divides the image to be identified into blocks according to a preset regional grid, and uses a block recognition model to identify individual blocks. Simultaneously, a batch recognition model is used to identify the extracted target regions in batches, significantly reducing the computational load for each identification. This method effectively shortens the identification time and improves the detection speed. Dynamically selecting a suitable batch recognition model based on the number of target regions to be identified further optimizes the identification process and improves overall identification efficiency. Both the image to be identified and the image regions to be identified maintain their original resolution, ensuring the accuracy of the identification results. By optimizing the identification process and model selection, the identification time is significantly shortened. While maintaining a certain level of detection accuracy, it also has a high detection speed, meeting the requirements for real-time / near real-time image target recognition on embedded devices.

[0116] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. For example... Figure 8As shown, the electronic device may include: a processor 110, a communication interface 120, a memory 130, and a communication bus 140, wherein the processor 110, the communication interface 120, and the memory 130 communicate with each other through the communication bus 140. The processor 110 can call logical instructions in the memory 130 to execute a target recognition method based on panoramic stitched images. The method includes: for an image to be recognized acquired at any time, dividing the image to be recognized into blocks according to a preset region grid to obtain the image region to be recognized at the given time; using a block recognition model to recognize the image region to be recognized, obtaining all recognition results output by the block recognition model; expanding each recognition result to obtain all current target regions corresponding to the given time; determining all target regions to be recognized based on all recognized regions corresponding to all preset times before the given time and all current target regions; and determining the target regions to be recognized based on the number of target regions to be recognized. A target batch recognition model is determined, and all target regions to be recognized are identified using the target batch recognition model to obtain all target recognition objects output by the target batch recognition model. All target recognition objects are then displayed in the image to be recognized. Each moment corresponds to processing a block in the preset region grid, and the sum of the number of moments of all preset moments and the current moment is the same as the number of blocks in the preset region grid. All recognized regions are determined by using the block recognition model to recognize the image to be recognized corresponding to each preset moment, obtaining all historical recognition results corresponding to each preset moment, and then expanding all historical recognition results corresponding to each preset moment to obtain all historical target regions corresponding to all preset moments.

[0117] Furthermore, the logical instructions in the aforementioned memory 130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a target recognition method based on panoramic stitched images provided by the above methods. The method includes: for an image to be recognized acquired at any time, dividing the image to be recognized into blocks according to a preset region grid to obtain the image region to be recognized at the time; using a block recognition model to recognize the image region to be recognized, obtaining all recognition results output by the block recognition model; expanding each recognition result to obtain all current target regions corresponding to the time; and based on all recognized regions corresponding to all preset times before the time and all current target regions, the method further divides the image region to be recognized into blocks according to a preset region grid to obtain all current target regions corresponding to the time. The target region is defined as follows: all target regions to be identified are determined; a target batch recognition model is determined based on the number of all target regions to be identified; the target batch recognition model is used to identify all target regions to be identified, and all target recognition objects output by the target batch recognition model are obtained; all target recognition objects are displayed in the image to be identified; wherein, each moment corresponds to processing a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid; all identified regions are determined by using the block recognition model to identify the image to be identified corresponding to each preset moment, obtaining all historical recognition results corresponding to each preset moment, and then expanding all historical recognition results corresponding to each preset moment to obtain all historical target regions corresponding to all preset moments.

[0119] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the target recognition method based on panoramic stitched images provided by the methods described above. This method includes: for an image to be recognized acquired at any given time, dividing the image to be recognized into blocks according to a preset region grid to obtain the image region to be recognized corresponding to the given time; using a block recognition model to recognize the image region to be recognized, obtaining all recognition results output by the block recognition model; expanding each recognition result to obtain all current target regions corresponding to the given time; and determining all target regions to be recognized based on all recognized regions corresponding to all preset times prior to the given time and all current target regions. The domain; a target batch recognition model is determined based on the number of all target regions to be recognized, and the target batch recognition model is used to recognize all target regions to be recognized, thereby obtaining all target recognition objects output by the target batch recognition model, and displaying all target recognition objects in the image to be recognized; wherein, each moment corresponds to processing a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid; the all recognized regions are determined by using the block recognition model to recognize the image to be recognized corresponding to each preset moment, obtaining all historical recognition results corresponding to each preset moment, and expanding all historical recognition results corresponding to each preset moment to obtain all historical target regions corresponding to all preset moments.

[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target recognition method based on panoramic stitched images, characterized in that, include: For an image to be identified acquired at any given time, the image to be identified is divided into blocks according to a preset region grid to obtain the image region to be identified corresponding to that time. Using a block recognition model, the image region to be recognized is identified, and all recognition results output by the block recognition model are obtained. Each recognition result is expanded to obtain all current target regions corresponding to the time. Based on all recognized regions corresponding to all preset times before the time and all current target regions, all target regions to be recognized are determined. A target batch recognition model is determined based on the number of all target regions to be recognized. The target batch recognition model is used to recognize all target regions to be recognized, and all target recognition objects output by the target batch recognition model are obtained. All target recognition objects are displayed in the image to be recognized. Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

2. The target recognition method based on panoramic stitched images according to claim 1, characterized in that, Before dividing the image to be identified into blocks according to a preset region grid, the method further includes: Use a panoramic pod to acquire camera images from different angles; All camera images are stitched together to obtain a panoramic stitched image. Redundancy in the overlapping parts of the panoramic stitched image is removed to obtain the image to be identified.

3. The target recognition method based on panoramic stitched images according to claim 2, characterized in that, The preset area grid has 15 blocks. The cameras in different positions include the center camera, the left camera, the right camera, the top camera, and the bottom camera. The camera images acquired by each camera are divided into 3 blocks. The step of dividing the image to be identified into blocks according to a preset region grid to obtain the image region to be identified at the specified time includes: The image to be identified is divided into blocks according to a preset region grid, and the block region corresponding to the block at the given time is obtained. The segmented region is determined as the image region to be identified at the time specified in the image to be identified.

4. The target recognition method based on panoramic stitched images according to claim 1, characterized in that, Before using a block recognition model to identify the image region to be identified and obtaining all recognition results output by the block recognition model, the method further includes: The first initial recognition model is trained using the original sample images corresponding to the original resolution and the sample recognition results corresponding to each original sample image to obtain the block recognition model; Both the image to be identified and the region of the image to be identified are at their original resolution.

5. The target recognition method based on panoramic stitched images according to claim 4, characterized in that, Before using the target batch recognition model to identify all the target regions to be identified, the method further includes: The second initial recognition model is trained using sample recognition images corresponding to a preset resolution and the sample recognition object corresponding to each sample recognition image to obtain the target batch recognition model; The preset resolution is smaller than the original resolution.

6. The target recognition method based on panoramic stitched images according to claim 1, characterized in that, The step of expanding each recognition result to obtain all current target regions corresponding to the given time includes: Determine the center point of the recognition result, and make the center point of the matrix region of a preset size coincide with the center point of the recognition result; The matrix region of the preset size is determined as the current target region corresponding to the recognition result; By iterating through all recognition results, all current target regions corresponding to the given time are obtained.

7. The target recognition method based on panoramic stitched images according to claim 1, characterized in that, The step of determining the target batch recognition model based on the number of all target regions to be identified includes: If the number of all target regions to be identified is less than or equal to 8, the target batch identification model is determined to be an 8-batch identification model. If the number of all target regions to be identified is greater than 8 and less than or equal to 16, the target batch identification model is determined to be a 16-batch identification model. If the number of all target regions to be identified is greater than 16 and less than or equal to 32, the target batch identification model is determined to be a 32-batch identification model. If the number of all target regions to be identified is greater than 32 and less than or equal to 64, the target batch identification model is determined to be a 64-batch identification model. If the number of all target regions to be identified is greater than 64, the target batch identification model is determined to be a 128-batch identification model.

8. The target recognition method based on panoramic stitched images according to claim 1, characterized in that, The target object to be identified is a vehicle. Before displaying all target objects in the image to be identified, the method further includes: For any target region to be identified, if more than one target object is identified, all target objects in the target region to be identified are filtered out, and only one target object is retained.

9. A target recognition device based on panoramic stitched images, characterized in that, include: The segmentation unit is used to segment the image to be identified according to a preset region grid for an image to be identified acquired at any time, so as to obtain the image region to be identified corresponding to the time in the image to be identified; The recognition unit is used to recognize the image region to be recognized using a block recognition model, obtain all recognition results output by the block recognition model, expand each recognition result to obtain all current target regions corresponding to the time, and determine all target regions to be recognized based on all recognized regions corresponding to all preset times before the time and all current target regions. A determining unit is configured to determine a target batch recognition model based on the number of all target regions to be identified, use the target batch recognition model to identify all target regions to be identified, obtain all target recognition objects output by the target batch recognition model, and display all target recognition objects in the image to be identified. Each moment corresponds to a block in the preset region grid, and the sum of the number of moments of all preset moments and the moment is the same as the number of blocks in the preset region grid. All identified regions are determined by using the block recognition model to identify the image to be identified at each preset moment, obtaining all historical recognition results at each preset moment, and then expanding all historical recognition results at each preset moment to obtain all historical target regions at all preset moments.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the target recognition method based on panoramic stitched images as described in any one of claims 1 to 8.

Images