A target localization method and apparatus, electronic device and storage medium
By acquiring the original image from the fisheye camera and ground reference pixels, semantic segmentation and distance calculation are performed, solving the problems of low target detection efficiency and inaccurate grounding points caused by fisheye camera image distortion, and achieving high-precision target localization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MUSHROOM CHELIAN INFORMATION TECH CO LTD
- Filing Date
- 2023-05-23
- Publication Date
- 2026-05-26
AI Technical Summary
Images captured by fisheye cameras suffer from severe distortion. Existing technologies are inefficient in target detection and localization, have difficulty recognizing edge targets, and cannot accurately identify the target's grounding point using calibration parameters.
By acquiring the original image from the fisheye camera and ground reference pixels, semantic segmentation is performed to determine the set of grounding pixels of the target, and the actual grounding point is calculated using distance relationships and calibration parameters.
It improves the accuracy and precision of the target grounding point, avoids the defects of distortion removal processing, and improves the calculation efficiency of target positioning.
Smart Images
Figure CN116580095B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a target localization method and apparatus, electronic device and storage medium. Background Technology
[0002] A fisheye camera is a commonly used camera with a wide-angle lens. The raw images captured by a fisheye camera suffer from severe distortion, such as... Figure 1 As shown, objects in the center of the original image maintain their original pose with minimal distortion. Objects in other locations outside the center of the image undergo some degree of pose change. For example, four pedestrians appear upside down in the image, indicating that there is significant distortion in the original image taken by the fisheye camera.
[0003] When performing target detection on images acquired using a fisheye camera, the conventional approach is to perform distortion correction on the original image, then perform target detection on the corrected image, and finally locate the target based on the detection results. Research has found that this approach has at least the following drawbacks:
[0004] First, an additional distortion correction step is required, which affects the efficiency of target detection and localization.
[0005] Second, distortion correction generally requires stretching the image, which can cause targets at the edges to be unable to be effectively identified.
[0006] Third, the calibration parameters of fisheye cameras are usually used to realize the transformation between the image coordinate system and the ground in the world coordinate system. The target-based rectangular detection box cannot accurately identify the grounding point of the target. Whether it is the center of the bottom edge, the center of the top edge, or other fixed points on the detection box, they may not be the actual grounding point of the target.
[0007] Therefore, it is necessary to propose a scheme that can accurately identify the actual grounding point of the target. Summary of the Invention
[0008] This application provides a target positioning method and apparatus, electronic device and storage medium to accurately identify the true grounding point of a target.
[0009] The embodiments of this application adopt the following technical solutions:
[0010] In a first aspect, embodiments of this application provide a target localization method, the method comprising:
[0011] Acquire the original image captured by the fisheye camera and the ground reference pixels on the original image;
[0012] The original image is semantically segmented to obtain the semantic segmentation result of each target, and a set of candidate pixels for calculating the grounding pixel of the target is obtained based on the semantic segmentation result of the target.
[0013] The grounding pixel of the target is obtained based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0014] The actual grounding point of the target is obtained based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0015] Optionally, obtaining the candidate pixel set for calculating the grounding pixel of the target based on the semantic segmentation result of the target includes:
[0016] Obtain the target category of the target, and determine the grounding location of the target based on the target category;
[0017] Determine whether the semantic segmentation result of the target includes the grounding part;
[0018] A set of candidate pixels for the target is constructed based on whether the semantic segmentation region of the target includes the grounding region.
[0019] Optionally, determining whether the semantic segmentation part included in the semantic segmentation result of the target includes the grounding part includes:
[0020] When the semantic segmentation part of the target includes the grounding part, a candidate pixel set of the target is constructed based on the pixels of the grounding part;
[0021] When the semantic segmentation region of the target does not include the grounding region, target detection is performed on the original image, and a set of candidate pixels of the target is constructed based on the target's detection box and the target's semantic segmentation result.
[0022] Optionally, constructing a set of candidate pixels for the target based on the target's detection bounding box and the target's semantic segmentation result includes:
[0023] A candidate pixel set for the target is constructed using preset pixels on the target's detection bounding box and all pixels from the semantic segmentation result.
[0024] Optionally, obtaining the grounding pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel includes:
[0025] Calculate the distance between each candidate pixel and the ground reference pixel, and select the candidate pixel with the smallest distance as the ground pixel of the target.
[0026] Optionally, ground reference pixels on the original image can be obtained through the following steps:
[0027] Determine the installation angle of the fisheye camera;
[0028] The ground reference pixels on the original image are obtained based on the installation angle of the fisheye camera.
[0029] Optionally, the installation angle includes a top-down installation angle, and obtaining the ground reference pixel on the original image based on the installation angle of the fisheye camera includes:
[0030] Obtain the image center point of the original image;
[0031] The center point of the image is used as the ground reference pixel.
[0032] Secondly, embodiments of this application also provide a target positioning device, the device comprising:
[0033] The data acquisition unit is used to acquire the original image captured by the fisheye camera and the ground reference pixels on the original image;
[0034] An image processing unit is used to perform semantic segmentation on the original image to obtain the semantic segmentation result of each target, and to obtain a set of candidate pixels for calculating the grounding pixels of the target based on the semantic segmentation result of the target.
[0035] A pixel calculation unit is used to obtain the ground pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0036] The grounding point determination unit is used to obtain the actual grounding point of the target based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0037] Thirdly, embodiments of this application also provide an electronic device, including:
[0038] Processor; and
[0039] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the aforementioned target localization methods.
[0040] Fourthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the aforementioned target location methods.
[0041] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:
[0042] In this embodiment, ground reference pixels are determined from the original image and the real ground is calibrated using the ground reference pixels. During the detection of the target's real grounding point, the distance relationship between the ground reference pixels and the target's grounding point is used to determine the target's real grounding point, thereby improving the accuracy and precision of the real grounding point.
[0043] On the other hand, the embodiments of this application also perform semantic segmentation on the original image, and obtain a set of candidate pixels for calculating the target grounding pixel based on the semantic segmentation results. This not only avoids the problem of unreliable target localization results caused by the detection box algorithm, but also improves the calculation efficiency of the target localization point.
[0044] Furthermore, the embodiments of this application perform target recognition on the original images from the fisheye camera, which does not require distortion correction processing and can avoid the related defects caused by distortion correction processing. Attached Figure Description
[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0046] Figure 1 A schematic diagram of the original image from a fisheye camera;
[0047] Figure 2 This is a flowchart illustrating a target localization method in an embodiment of this application;
[0048] Figure 3 This is a schematic diagram of the structure of a target positioning device according to an embodiment of this application;
[0049] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0051] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0052] This application provides a target localization method. The following embodiments use a roadside device as the execution subject of the target localization method for illustrative purposes. It is understood that the execution subject of this application's embodiments can also be other devices or systems, such as a roadside edge computing system, an autonomous driving system, or a chip system. A chip system includes at least one processor, and a communication interface or memory, wherein the memory is used to store programs or instructions, and the communication interface is used to provide information or data input / output functions to the processor. Alternatively, the execution subject of the target localization method in this application's embodiments can also be software or hardware, and different execution steps can be centralized or distributed.
[0053] like Figure 2 The diagram shows a flowchart of a target localization method according to an embodiment of this application. The method includes at least the following steps S210 to S240:
[0054] Step S210: Obtain the original image captured by the fisheye camera and the ground reference pixels on the original image.
[0055] In this embodiment of the application, after acquiring the original image through a fisheye camera, it is also necessary to obtain ground reference pixels from the original image. Ground reference pixels are pixels used to calibrate the ground in the real world. In this way, during the target localization process, the ground reference pixels can be used as a reference ground. By determining the relationship between the target pixels and the ground reference pixels, the accuracy of the identified target's true grounding point can be improved.
[0056] like Figure 1 As can be seen from the original image shown, Figure 1 In this image, the part of each pedestrian closest to the center of the image is the pedestrian's foot. In other words, there is a certain distance relationship between the pixel of the target's grounding point and the ground reference pixel. Therefore, this embodiment of the application uses the ground reference pixel on the original image to determine the target's real grounding point based on this objective law, thereby improving the recognition accuracy.
[0057] Step S220: Perform semantic segmentation on the original image to obtain the semantic segmentation result of each target, and obtain the candidate pixel set for calculating the ground pixel of the target based on the semantic segmentation result of the target.
[0058] Continue to refer to Figure 1 In the raw images from a fisheye camera, the poses of various targets outside the center of the image are offset to some extent. In this case, although some target detection algorithms based on detection boxes can correct the target's deflection by rotating the detection box during the detection process, there will still be cases where the target's contact point is not located on the bottom edge or other sides of the detection box, thus failing to obtain the target's accurate contact point based on the detection box.
[0059] In response to the above situation, the embodiments of this application perform semantic segmentation on the original image, and the obtained semantic segmentation result includes all pixels belonging to the target, and determines a set of candidate pixels from all pixels of the target.
[0060] Referring to the preceding text, it is known that there is a certain distance relationship between the pixels at the target's grounding point and the ground reference pixels. Therefore, in this embodiment, the grounding point of the target can be predefined according to the target type, for example, the grounding point of a pedestrian is defined as their feet, and the grounding point of a vehicle is defined as its wheels. After obtaining the semantic segmentation result of the target, a set of candidate pixels that have the aforementioned distance relationship with the ground reference pixels can be determined based on the pixel type of the target. The grounding pixels of the target can then be determined from this set of candidate pixels. This approach avoids interference from other parts of the target with the target localization result, improving the accuracy of the grounding pixels. Furthermore, it reduces the number of pixels involved in the calculation of the grounding pixels, thus improving computational efficiency.
[0061] Step S230: Obtain the grounding pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0062] Step S240: Obtain the actual grounding point of the target based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0063] like Figure 2As shown in the target localization method, this embodiment of the application, on the one hand, determines ground reference pixels from the original image and uses these ground reference pixels to calibrate the real ground. During the detection of the target's actual grounding point, the distance relationship between the ground reference pixels and the target's grounding point is used to determine the target's actual grounding point, thereby improving the accuracy and precision of the actual grounding point. On the other hand, this embodiment of the application also performs semantic segmentation on the original image, obtaining a set of candidate pixels for calculating the target's grounding point based on the semantic segmentation results. This not only avoids the problem of unreliable target localization results caused by detection box algorithms but also improves the computational efficiency of target localization points. Furthermore, this embodiment of the application performs target recognition on the original image from a fisheye camera, eliminating the need for distortion correction processing and avoiding the related defects caused by distortion correction processing.
[0064] In some embodiments of this application, the specific steps for obtaining the ground reference pixel points on the original image in step S210 are as follows:
[0065] Determine the installation angle of the fisheye camera;
[0066] The ground reference pixels on the original image are obtained based on the installation angle of the fisheye camera.
[0067] In some implementations of this embodiment, when the fisheye lens is mounted at a top-down angle, obtaining the ground reference pixel on the original image based on the fisheye lens's mounting angle specifically includes:
[0068] Obtain the center point of the original image and use the center point as the ground reference pixel.
[0069] The position of the ground reference pixels in the original image varies depending on the installation angle of the fisheye camera. For example, when the fisheye camera is installed with a downward view relative to the ground, the position of the ground reference pixels will differ. Figure 1 As shown, in a roadside scene, the center point of the original image is the ground reference pixel. However, when a fisheye camera is mounted at a certain angle relative to the ground, such as in a vehicle-mounted scene, the ground reference pixel of the original image will be offset from the center point of the image by a certain angle. Those skilled in the art can determine the ground reference pixel based on the mounting angle of the vehicle-mounted fisheye camera.
[0070] In some embodiments of this application, the step S220 above, which obtains a set of candidate pixels for calculating the grounding pixels of the target based on the semantic segmentation result of the target, specifically includes:
[0071] Obtain the target category of the target, and determine the grounding location of the target based on the target category;
[0072] Determine whether the semantic segmentation result of the target includes the grounding part;
[0073] A candidate pixel set for the target is constructed based on whether the semantic segmentation region of the target includes the grounding region. Specifically, when the semantic segmentation region of the target includes the grounding region, the candidate pixel set for the target is constructed based on the pixels of the grounding region; when the semantic segmentation region of the target does not include the grounding region, target detection is performed on the original image, and the candidate pixel set for the target is constructed based on the target detection box and the semantic segmentation result of the target.
[0074] This embodiment constructs a candidate pixel set based on whether the semantic segmentation region includes a grounding region. The reason for constructing the candidate pixel set based on the grounding region is that, in some cases, other parts of the target and ground reference pixels may also conform to the aforementioned distance relationship in a single frame of the original image. For example, during a pedestrian's walk, the swinging arm and ground reference pixels may also conform to the distance relationship described above. To address this issue, this embodiment determines whether the semantic segmentation region includes a grounding region; only when a grounding region is included are the target's grounding pixels calculated from the pixels of the grounding region.
[0075] Furthermore, semantic segmentation of images involves classifying pixels according to a specific semantic segmentation algorithm. When certain targets in an image are occluded, the pixels of the occluded portion will not be classified as occluded target pixels. Therefore, in some scenarios, when the grounding portion of a target is occluded, directly calculating the candidate pixel set based on the semantic segmentation results of the target will result in poor accuracy and low precision of the final calculated grounding pixels, making the target localization results unreliable.
[0076] Therefore, after obtaining the semantic segmentation result of the target, this embodiment further determines the grounding part of the target based on the target category. In practical applications, grounding parts corresponding to various types of targets can be pre-set, such as the foot for a pedestrian and the wheel for a vehicle. When the semantic segmentation part of the target includes the grounding part, the pixels of the grounding part can be directly used as candidate pixels. When the semantic segmentation part of the target does not include the grounding part, this embodiment performs target detection on the original image to obtain the target detection box, and combines the detection box and the semantic segmentation result to construct a candidate pixel set.
[0077] The advantage of bounding box-based target detection algorithms is that occluded parts of the target can still be detected and classified into the bounding box. Therefore, when the semantic segmentation does not include the grounding part of the target, this embodiment constructs a candidate pixel set by combining the target's bounding box and the semantic segmentation result. Specifically, the candidate pixel set is constructed using preset pixels on the target's bounding box and all pixels from the semantic segmentation result. For example, in some scenarios, the center pixels of the four sides of the bounding box and all pixels from the semantic segmentation result can be used as candidate pixels. Of course, in other scenarios, the vertices and / or the center pixels of the four sides of the bounding box and all pixels from the semantic segmentation result can also be used as candidate pixels.
[0078] Thus, after obtaining the candidate pixel set of the target through the above embodiments, the step S230 above, which obtains the ground pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel, specifically includes:
[0079] Calculate the distance between each candidate pixel and the ground reference pixel, and select the candidate pixel with the smallest distance as the ground pixel of the target.
[0080] This embodiment can calculate the Euclidean distance between pixels. Of course, other distance algorithms can also be used for calculation, and those skilled in the art can choose flexibly.
[0081] It should be understood that the embodiments of this application illustrate an implementation scheme for calculating the ground pixel of a target based on the distance relationship between candidate pixels and ground reference pixels. In other embodiments of this application, other implementation schemes may also be adopted, such as calculating the contour edge corresponding to the candidate pixel set, calculating the distance between the contour edge pixels and the ground reference pixels, and taking the contour edge pixels corresponding to the minimum distance as the ground pixel of the target.
[0082] After obtaining the grounding pixels of the target, the actual grounding point of the target can be obtained based on the grounding pixels and the calibration parameters of the fisheye camera. In this embodiment, the calibration parameters of the fisheye camera are used to realize the transformation between the image coordinate system and the ground-based world coordinate system. Those skilled in the art can refer to existing technologies for the implementation scheme of coordinate transformation of the target's grounding pixels based on the calibration parameters, and it will not be described in detail here.
[0083] This application embodiment also provides a target positioning device 300, such as Figure 3 The diagram shows a target positioning device according to an embodiment of this application. The device 300 includes: a data acquisition unit 310, an image processing unit 320, a pixel calculation unit 330, and a grounding point determination unit 340, wherein:
[0084] The data acquisition unit 310 is used to acquire the original image captured by the fisheye camera and the ground reference pixels on the original image;
[0085] Image processing unit 320 is used to perform semantic segmentation on the original image to obtain the semantic segmentation result of each target, and to obtain a set of candidate pixels for calculating the grounding pixels of the target based on the semantic segmentation result of the target.
[0086] The pixel calculation unit 330 is used to obtain the ground pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0087] The grounding point determination unit 340 is used to obtain the actual grounding point of the target based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0088] In some embodiments of this application, the image processing unit 320 is specifically used to obtain the target category of the target, and determine the grounding part of the target according to the target category; determine whether the semantic segmentation part included in the semantic segmentation result of the target includes the grounding part; and construct a set of candidate pixels of the target according to whether the semantic segmentation part of the target includes the grounding part.
[0089] In some embodiments of this application, the image processing unit 320 is specifically configured to construct a candidate pixel set of the target based on the pixels of the grounding part when the semantic segmentation part of the target includes the grounding part; and to perform target detection on the original image and construct the candidate pixel set of the target based on the detection box of the target and the semantic segmentation result of the target when the semantic segmentation part of the target does not include the grounding part.
[0090] In some embodiments of this application, the image processing unit 320 is specifically used to construct a set of candidate pixels of the target using preset pixels on the detection box of the target and all pixels of the semantic segmentation result.
[0091] In some embodiments of this application, the pixel calculation unit 330 is specifically used to calculate the distance between each candidate pixel and the ground reference pixel, and to take the candidate pixel with the smallest distance as the ground pixel of the target.
[0092] In some embodiments of this application, the data acquisition unit 310 is specifically used to determine the installation angle of the fisheye camera; and to acquire ground reference pixels on the original image based on the installation angle of the fisheye camera.
[0093] In some embodiments of this application, the installation angle includes a top-down installation angle, and the data acquisition unit 310 is specifically used to acquire the image center point of the original image and use the image center point as the ground reference pixel point.
[0094] It is understood that the target positioning device described above can implement each step of the target positioning method provided in the foregoing embodiments. The relevant explanations of the target positioning method are applicable to the target positioning device and will not be repeated here.
[0095] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.
[0096] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.
[0097] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.
[0098] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a target location device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:
[0099] Acquire the original image captured by the fisheye camera and the ground reference pixels on the original image;
[0100] The original image is semantically segmented to obtain the semantic segmentation result of each target, and a set of candidate pixels for calculating the grounding pixel of the target is obtained based on the semantic segmentation result of the target.
[0101] The grounding pixel of the target is obtained based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0102] The actual grounding point of the target is obtained based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0103] The above is as stated in this application. Figure 2 The target positioning device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads the information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned target localization method.
[0104] The electronic device can also perform Figure 2 The method executed by the target positioning device, and the implementation of the target positioning device in Figure 2 The functions of the embodiments shown are not described in detail here.
[0105] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 2 The method executed by the target positioning device in the illustrated embodiment is specifically used to perform:
[0106] Acquire the original image captured by the fisheye camera and the ground reference pixels on the original image;
[0107] The original image is semantically segmented to obtain the semantic segmentation result of each target, and a set of candidate pixels for calculating the grounding pixel of the target is obtained based on the semantic segmentation result of the target.
[0108] The grounding pixel of the target is obtained based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel.
[0109] The actual grounding point of the target is obtained based on the grounding pixel points of the target and the calibration parameters of the fisheye camera.
[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0111] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0114] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0115] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0116] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0117] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0119] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A target localization method, characterized in that, The method includes: Acquire the original image captured by the fisheye camera and the ground reference pixels on the original image; The original image is semantically segmented to obtain the semantic segmentation result of each target, and a set of candidate pixels for calculating the grounding pixel of the target is obtained based on the semantic segmentation result of the target. The grounding pixel of the target is obtained based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel. The actual grounding point of the target is obtained based on the grounding pixel points of the target and the calibration parameters of the fisheye camera; The step of obtaining the candidate pixel set for calculating the grounding pixel of the target based on the semantic segmentation result of the target includes: Obtain the target category of the target, and determine the grounding location of the target based on the target category; Determine whether the semantic segmentation result of the target includes the grounding part; A set of candidate pixels for the target is constructed based on whether the semantic segmentation part of the target includes the grounding part; Whether the semantic segmentation part included in the semantic segmentation result of the target includes the grounding part is determined as follows: When the semantic segmentation part of the target includes the grounding part, a candidate pixel set of the target is constructed based on the pixels of the grounding part; When the semantic segmentation region of the target does not include the grounding region, target detection is performed on the original image, and a set of candidate pixels of the target is constructed based on the target's detection box and the target's semantic segmentation result.
2. The method as described in claim 1, characterized in that, The step of constructing a candidate pixel set for the target based on the target's detection bounding box and the target's semantic segmentation result includes: A candidate pixel set for the target is constructed using preset pixels on the target's detection bounding box and all pixels from the semantic segmentation result.
3. The method as described in claim 1, characterized in that, The step of obtaining the grounding pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel includes: Calculate the distance between each candidate pixel and the ground reference pixel, and select the candidate pixel with the smallest distance as the ground pixel of the target.
4. The method as described in claim 1, characterized in that, The ground reference pixels on the original image are obtained through the following steps: Determine the installation angle of the fisheye camera; The ground reference pixels on the original image are obtained based on the installation angle of the fisheye camera.
5. The method as described in claim 4, characterized in that, The installation angle includes a top-down installation angle, and obtaining the ground reference pixel point on the original image based on the installation angle of the fisheye camera includes: Obtain the image center point of the original image; The center point of the image is used as the ground reference pixel.
6. A target positioning device, characterized in that, The device includes: The data acquisition unit is used to acquire the original image captured by the fisheye camera and the ground reference pixels on the original image; An image processing unit is used to perform semantic segmentation on the original image to obtain the semantic segmentation result of each target, and to obtain a set of candidate pixels for calculating the grounding pixels of the target based on the semantic segmentation result of the target. A pixel calculation unit is used to obtain the ground pixel of the target based on the distance relationship between each candidate pixel in the candidate pixel set and the ground reference pixel. The grounding point determination unit is used to obtain the actual grounding point of the target based on the grounding pixel points of the target and the calibration parameters of the fisheye camera; The image processing unit is specifically used for: Obtain the target category of the target, and determine the grounding location of the target based on the target category; Determine whether the semantic segmentation result of the target includes the grounding part; A set of candidate pixels for the target is constructed based on whether the semantic segmentation part of the target includes the grounding part; The image processing unit is specifically used for: When the semantic segmentation part of the target includes the grounding part, a candidate pixel set of the target is constructed based on the pixels of the grounding part; When the semantic segmentation region of the target does not include the grounding region, target detection is performed on the original image, and a set of candidate pixels of the target is constructed based on the target's detection box and the target's semantic segmentation result.
7. An electronic device, characterized in that, include: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the target localization method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform the target localization method according to any one of claims 1 to 5.