Target detection method and device, electronic equipment and storage medium
By using a convolutional neural network model in a fisheye camera to detect targets and correct positional offsets, the problem of inaccurate positioning in fisheye images is solved, improving the accuracy and reliability of target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2023-01-05
- Publication Date
- 2026-06-19
AI Technical Summary
Fisheye cameras suffer from distortion and positioning point deviation in target detection, leading to inaccurate positioning.
A convolutional neural network model is used to detect target bounding boxes in fisheye images. The position points in the target bounding boxes are corrected by determining the position offset of the target. The position offset is calculated using the center point of the fisheye image and the center point of the target bounding box to redetermine the target localization result.
It improves the reliability of target localization in fisheye images, reduces localization deviation, ensures the accuracy of target 3D projection, and reduces misjudgment in non-road driving areas.
Smart Images
Figure CN116051812B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to a target detection method, device, electronic device, and storage medium. Background Technology
[0002] Fisheye cameras have an extremely wide field of view, making them advantageous for detecting wide-angle, short-distance scenes. Typically, fisheye cameras are fixed to roadside poles as part of roadside equipment, used to detect an area of approximately 20 meters in front of and behind the pole. They can also be used in conjunction with near-field and far-field cameras to track targets on the road.
[0003] In related technologies, when using a fisheye camera to detect targets, it is necessary not only to consider the distortion caused by fisheye imaging, but also to overcome the problem of deviation in the positioning points within the fisheye image. Furthermore, automatic correction of the target positioning points is required for different areas where the target appears. Summary of the Invention
[0004] This application provides a target detection method, apparatus, electronic device, and storage medium to correct target positioning points in fisheye images and improve positioning reliability.
[0005] The embodiments of this application adopt the following technical solutions:
[0006] In a first aspect, embodiments of this application provide a target detection method, wherein the method includes:
[0007] Acquire fisheye images captured by a fisheye camera;
[0008] Based on the target detection model, the target detection box of the target in the fisheye image is obtained;
[0009] The position offset of the target is determined based on the first position point in the fisheye image and the second position point in the target detection box;
[0010] The second position point in the target detection box is corrected based on the position offset of the target to obtain the third position point in the target detection box, and the third position point is used as the positioning result of the target in the fisheye image.
[0011] In some embodiments, correcting the second position point in the target detection box based on the position offset of the target to obtain a third position point in the target detection box, and using the third position point as the positioning result of the target in the fisheye image, includes:
[0012] The second location point in the target detection box is used as the initial positioning result of the target in the fisheye image;
[0013] The initial positioning result in the fisheye image is corrected based on the position offset of the target to obtain the final positioning result of the target in the fisheye image, wherein the final positioning result of the target in the fisheye image is the third position point re-determined in the target detection box.
[0014] In some embodiments, the first location point includes the center point of the fisheye image, and the second location point includes the center point of the target detection box.
[0015] In some embodiments, determining the position offset of the target based on a first position point in the fisheye image and a second position point in the target detection box includes:
[0016] The relative position of the target in the fisheye image is determined based on the center point of the fisheye image and the center point of the target detection box.
[0017] Based on the relative positional relationship, the positional offset of the target is determined.
[0018] In some embodiments, the position offset includes at least the offset amount, the offset direction, and the correction offset.
[0019] In some embodiments, acquiring the image captured by the fisheye camera includes:
[0020] Obtain a fisheye image in which the Y-axis of the imaging plane of the fisheye camera is parallel to the road surface.
[0021] The fisheye image is subjected to distortion correction processing to obtain the image directly below the fisheye camera.
[0022] In some embodiments, obtaining the target detection bounding box of the target in the image according to the target detection model includes:
[0023] The object detection model based on convolutional neural networks detects objects in the image and outputs the object detection box.
[0024] Secondly, embodiments of this application also provide a target detection device, wherein the device includes:
[0025] The acquisition module is used to acquire fisheye images captured by the fisheye camera;
[0026] The detection module is used to obtain the target detection box of the target in the fisheye image according to the target detection model;
[0027] The determination module is used to determine the position offset of the target based on a first position point in the fisheye image and a second position point in the target detection box;
[0028] The correction module is used to correct the second position point in the target detection box according to the position offset of the target, obtain the third position point in the target detection box, and use the third position point as the positioning result of the target in the fisheye image.
[0029] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.
[0030] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.
[0031] The at least one technical solution adopted in this application can achieve the following beneficial effects: acquiring a fisheye image captured by a fisheye camera, and then detecting the target detection box of the target in the fisheye image through a target detection model. Since there is a positioning deviation in the fisheye image, it is necessary to determine the target's position offset and correct the position point in the target detection box based on the target's position offset, thereby correcting the target positioning results at different positions in the fisheye image. This application reduces positioning deviation and improves positioning reliability by correcting the position point in the target detection box of the target detection model in the fisheye image and using the corrected position point as the current target's positioning point in the fisheye image. Furthermore, the result of 3D projection of the target based on the corrected 2D target position point is more accurate, reducing the occurrence of projections into non-road driving areas. Attached Figure Description
[0032] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0033] Figure 1 This is a schematic diagram of the target detection method in the embodiments of this application;
[0034] Figure 2(a) is a schematic diagram of the original fisheye image in the fisheye camera in the target detection method of this application embodiment;
[0035] Figure 2(b) is a schematic diagram of the corrected image in the fisheye camera in an embodiment of this application;
[0036] Figure 3 This is a schematic diagram of the redefined positioning points in the target detection frame in the embodiments of this application;
[0037] Figure 4This is a schematic diagram of the target detection device structure in the embodiments of this application;
[0038] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0040] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0041] This application provides a target detection method, such as... Figure 1 The diagram shows a flowchart of a target detection method in an embodiment of this application. The method includes at least the following steps S110 to S140:
[0042] Step S110: Obtain the fisheye image captured by the fisheye camera.
[0043] Fisheye cameras need to be installed face down, and the fisheye image captured by the camera is the image directly below. Fisheye images are susceptible to image distortion, so distortion correction must be performed before detection.
[0044] It is understood that there are various methods for image distortion correction, with checkerboard calibration and longitude correction being commonly used. In this embodiment, target detection only requires correction of the radial (Y-axis) distortion of the fisheye image to overcome the positioning deviation caused by radial distortion. Preferably, longitude correction is used for fisheye image distortion correction.
[0045] Fisheye cameras can work in conjunction with close-up and long-range cameras mounted on roadside poles, which are also installed as roadside equipment, to locate and track targets. Furthermore, fisheye cameras, close-up cameras, and long-range cameras have different monitoring areas or sensing ranges.
[0046] Step S120: Obtain the target detection box of the target in the fisheye image according to the target detection model.
[0047] The target detection model mainly considers a model trained with a convolutional neural network, which can perform target detection in the fisheye image after distortion correction and output the target detection bounding box.
[0048] The target detection model used is the YOLOv7-tiny model, trained on a training set obtained through image annotation. This ensures both detection speed and accuracy. As a real-time target detector, the YOLOv7-tiny model is designed for edge devices and GPU architectures. It uses ReLU as the activation function. Compared to the current best model, the YOLOv7-tiny model has fewer parameters and lower computational cost, resulting in faster inference speed and higher detection accuracy. In other words, the YOLOv7-tiny model is highly suitable for fisheye cameras on roadside devices due to its small parameter count and low computational cost, reducing the computational demands on the roadside equipment and ensuring real-time detection.
[0049] After detection by the YOLOv7-tiny model and post-processing by NMS, the target detection bounding box can be output.
[0050] It's important to note that the object detection boxes here are 2D object detection boxes. NMS (Non-Maximum Suppression) is widely used in many computer vision tasks, such as edge detection and object detection. In the embodiments of this application, it is mainly used for object detection. For example, when locating a vehicle in a fisheye image, after the model identifies some boxes (detection boxes), it's necessary to determine which boxes are useless. That is, after identifying multiple object detection boxes in the fisheye image using the object detection model, NMS is needed to eliminate redundant boxes, thereby finding the optimal object detection location.
[0051] Step S130: Determine the position offset of the target based on the first position point in the fisheye image and the second position point in the target detection box.
[0052] When locating a target using the bounding box detected by the model in the fisheye image, the center point of the bounding box is usually taken as the target location point. However, experiments have shown that for targets with high height or a high center of gravity, using the center point of the bounding box as the target location point is often inaccurate, causing significant positioning errors. Taking a vehicle as an example, since the origin of the vehicle's coordinate system coincides with its center of gravity, the center point of the bounding box can be used as the target location point when the vehicle's center of gravity is low. However, if the vehicle's center of gravity is high, using the center point of the bounding box as the target location point will lead to inaccurate positioning.
[0053] Preferably, the first location point in the fisheye image is selected as the center point of the fisheye image, and the second location point in the target detection box is selected as the center point of the target detection box.
[0054] The positional offset of the target is determined based on the center point in the fisheye image and the center point in the target detection box.
[0055] Step S140: Correct the second position point in the target detection box according to the position offset of the target to obtain the third position point in the target detection box, and use the third position point as the positioning result of the target in the fisheye image.
[0056] Since the fisheye camera is fixed, and the direction of vehicle travel is roughly fixed in the fisheye image, the offset correction can be performed based on the relative position of the vehicle in the image, according to the center point, making the positioning point closer to the actual situation. That is, by using the relative position of the target at different positions in the fisheye camera, the position point that can be used as the target positioning point (the center point position of the target detection box) in the target detection box detected by the target detection model can be re-determined.
[0057] Since the center of the image is the center of the fisheye image, and the angle observed by the fisheye camera varies depending on the relative position, we can roughly determine the current angle of observation within the fisheye camera based on the relative position. For example, the observation angle at the center position is a top-down view. The center of the target detection box seen from a top-down view is accurate, but the center of the target detection box observed from other top-down angles in the fisheye image is inaccurate.
[0058] The third location point in the target detection box is a corrected location point, which is an accurate location point. Therefore, the third location point can be used as the location result of the target in the fisheye image.
[0059] In one embodiment of this application, the step of correcting the second position point in the target detection box according to the position offset of the target to obtain a third position point in the target detection box, and using the third position point as the positioning result of the target in the fisheye image, includes: using the second position point in the target detection box as the initial positioning result of the target in the fisheye image; correcting the initial positioning result in the fisheye image according to the position offset of the target to obtain the final positioning result of the target in the fisheye image, wherein the final positioning result of the target in the fisheye image is the re-determined third position point in the target detection box.
[0060] The second position point in the target detection box can be used as the initial positioning result of the target in the fisheye image. If the initial positioning result is the downward angle directly below the fisheye camera, then the initial positioning result does not need to be corrected and is accurate.
[0061] Similarly, if the initial positioning result is not a top-down view from directly below the fisheye camera, the fisheye image in the fisheye camera will be inaccurate due to the different observation angle. Therefore, it is necessary to correct the initial positioning result in the fisheye image based on the target's position offset to obtain the final positioning result of the target in the fisheye image.
[0062] The "target position offset" here is the offset between the pixel coordinates of the vehicle's initial positioning result in the fisheye image and the pixel coordinates of the center point in the fisheye image.
[0063] In one embodiment of this application, determining the position offset of the target based on a first position point in the fisheye image and a second position point in the target detection box includes: determining the relative positional relationship of the target in the fisheye image based on the center point of the fisheye image and the center point of the target detection box; and determining the position offset of the target based on the relative positional relationship.
[0064] Preferably, the position offset includes at least the offset amount, the offset direction, and the correction offset. The specific calculation process is described in detail below, where abs is the absolute coordinate system.
[0065] Set the center point of the fisheye image to (Ix, Iy);
[0066] The center point of the target detection box is (bx,by), and the width and height of the target detection box are (w,h).
[0067] The positional offset of the target detection box relative to the center of the image includes:
[0068] Offset:
[0069] dx = abs(bx - Ix)
[0070] dy = abs(by - Iy)
[0071] Offset direction:
[0072] fx = -dx / abs(dx)
[0073] fy = -dy / abs(dy)
[0074] Correcting bias:
[0075] delta_x = (w / 2)*(dx / Ix)^sigma
[0076] delta_y=(h / 2)*(dy / Iy)^sigma
[0077] Here, sigma is called the adjustment coefficient, which is the sensitivity of the correction bias to the relative center offset. The smaller the sigma, the larger the correction bias.
[0078] Finally, the coordinates of the new positioning point (third position point) are:
[0079] ox = bx + delta_x * fx
[0080] oy = by + delta_y * fy
[0081] In one embodiment of this application, acquiring the image captured by the fisheye camera includes: the Y-axis of the imaging plane in the fisheye camera is parallel to the road surface; and the fisheye image is subjected to distortion correction processing to obtain an image directly below the fisheye camera.
[0082] The fisheye camera is mounted facing downwards, with the Y-axis of its imaging plane parallel to the road surface. The fisheye image is shown in Figure 2(a). Therefore, the orientation of the target vehicle in the image is approximately the Y-direction of the image. This is achieved by obtaining a fisheye image where the Y-axis of the imaging plane is parallel to the road surface.
[0083] To overcome the influence of image distortion in fisheye images, distortion correction processing is performed on the fisheye images before detection, resulting in the corrected image shown in Figure 2(b), which is similar to a distortion-free image taken directly below a fisheye camera. In this embodiment, longitude correction is used for fisheye distortion correction.
[0084] The formula for longitude correction is:
[0085]
[0086] mapy[i, j] = i
[0087] Where mapx[i,j] represents the x value when the target image position [i,j] corresponds to the original image;
[0088] mapy[i,j] represents the y-value of the target image at position [i,j] when it corresponds to the original image;
[0089] R represents the radius of the sphere in the fisheye image.
[0090] In one embodiment of this application, obtaining the target detection box of the target in the image according to the target detection model includes: detecting the target in the image based on the target detection model of the convolutional neural network and outputting the target detection box.
[0091] When locating a target in a fisheye image using a model-detected bounding box, the center point of the bounding box is typically taken as the target location point. For example... Figure 3 As shown, the center point of the target detection box is used as the target's position in the fisheye image, and is represented by a "dark" point. The corrected target position point (the third position point) is represented by a "light" point. Since the third position point in the target detection box is an accurate, corrected location point, it can be used as the target's location result in the fisheye image.
[0092] Furthermore, after obtaining the new 2D location points, the actual 3D location of the target can be located through calibration projection mapping of the fisheye camera, which can be used for subsequent target localization and tracking.
[0093] This application embodiment also provides a target detection device 400, such as Figure 4 As shown, a schematic diagram of the target detection device in an embodiment of this application is provided. The target detection device 400 includes at least: an acquisition module 410, a detection module 420, a determination module 430, and a correction module 440, wherein:
[0094] In one embodiment of this application, the acquisition module 410 is specifically used to: acquire fisheye images captured by a fisheye camera.
[0095] Fisheye cameras need to be installed face down, and the fisheye image captured by the camera is the image directly below. Fisheye images are susceptible to image distortion, so distortion correction must be performed before detection.
[0096] It is understood that there are various methods for image distortion correction, with checkerboard calibration and longitude correction being commonly used. In this embodiment, target detection only requires correction of the radial (Y-axis) distortion of the fisheye image to overcome the positioning deviation caused by radial distortion. Preferably, longitude correction is used for fisheye image distortion correction.
[0097] Fisheye cameras can work in conjunction with close-up and long-range cameras mounted on roadside poles, which are also installed as roadside equipment, to locate and track targets. Furthermore, fisheye cameras, close-up cameras, and long-range cameras have different monitoring areas or sensing ranges.
[0098] In one embodiment of this application, the detection module 420 is specifically used to: obtain the target detection box of the target in the fisheye image according to the target detection model.
[0099] The target detection model mainly considers a model trained with a convolutional neural network, which can perform target detection in the fisheye image after distortion correction and output the target detection bounding box.
[0100] The target detection model used is the YOLOv7-tiny model, trained on a training set obtained through image annotation. This ensures both detection speed and accuracy. As a real-time target detector, the YOLOv7-tiny model is designed for edge devices and GPU architectures. It uses ReLU as the activation function. Compared to the current best model, the YOLOv7-tiny model has fewer parameters and lower computational cost, resulting in faster inference speed and higher detection accuracy. In other words, the YOLOv7-tiny model is highly suitable for fisheye cameras on roadside devices due to its small parameter count and low computational cost, reducing the computational demands on the roadside equipment and ensuring real-time detection.
[0101] After detection by the YOLOv7-tiny model and post-processing by NMS, the target detection bounding box can be output.
[0102] It's important to note that the object detection boxes here are 2D object detection boxes. NMS (Non-Maximum Suppression) is widely used in many computer vision tasks, such as edge detection and object detection. In the embodiments of this application, it is mainly used for object detection. For example, when locating a vehicle in a fisheye image, the model finds a bunch of boxes (detection boxes), and it's necessary to determine which boxes are useless. That is, after the object detection model identifies multiple object detection boxes in the fisheye image, NMS is needed to eliminate redundant boxes, thereby finding the optimal object detection location.
[0103] In one embodiment of this application, the determining module 430 is specifically used to: determine the position offset of the target based on a first position point in the fisheye image and a second position point in the target detection box.
[0104] When locating targets in a fisheye image using the target detection bounding box obtained from the model, the center point of the bounding box is usually taken as the target location point. However, experiments have shown that for targets with high height or a high center of gravity, using the center point of the bounding box as the target location point is often inaccurate, causing significant positioning errors. Taking a vehicle as an example, since the origin of the vehicle's coordinate system coincides with its center of gravity, the center point of the bounding box can be used as the target location point when the vehicle's center of gravity is low. However, if the vehicle's center of gravity is high, using the center point of the bounding box as the target location point will lead to inaccurate positioning.
[0105] Preferably, the first location point in the fisheye image is selected as the center point of the fisheye image, and the second location point in the target detection box is selected as the center point of the target detection box.
[0106] The positional offset of the target is determined based on the center point in the fisheye image and the center point in the target detection box.
[0107] In one embodiment of this application, the correction module 440 is specifically used to: correct the second position point in the target detection box according to the position offset of the target, obtain the third position point in the target detection box, and use the third position point as the positioning result of the target in the fisheye image.
[0108] Since the fisheye camera is fixed, and the direction of vehicle travel is roughly fixed in the fisheye image, the offset correction can be performed based on the relative position of the vehicle in the image, according to the center point, making the positioning point closer to the actual situation. That is, by using the relative position of the target at different positions in the fisheye camera, the position point that can be used as the target positioning point (the center point position of the target detection box) in the target detection box detected by the target detection model can be re-determined.
[0109] Since the center of the image is the center of the fisheye image, and the angle observed by the fisheye camera varies depending on the relative position, we can roughly determine the current angle of observation within the fisheye camera based on the relative position. For example, the observation angle at the center position is a top-down view. The center of the target detection box seen from a top-down view is accurate, but the center of the target detection box observed from other top-down angles in the fisheye image is inaccurate.
[0110] When the third location point in the target detection box is obtained, it is an accurate location point after correction, so the third location point can be used as the location result of the target in the fisheye image.
[0111] It is understood that the target detection device described above can implement each step of the target detection method provided in the foregoing embodiments. The relevant explanations of the target detection method are applicable to the target detection device and will not be repeated here.
[0112] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 5At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.
[0113] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.
[0114] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.
[0115] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a target detection device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:
[0116] Acquire fisheye images captured by a fisheye camera;
[0117] Based on the target detection model, the target detection box of the target in the fisheye image is obtained;
[0118] The position offset of the target is determined based on the first position point in the fisheye image and the second position point in the target detection box;
[0119] The second position point in the target detection box is corrected based on the position offset of the target to obtain the third position point in the target detection box, and the third position point is used as the positioning result of the target in the fisheye image.
[0120] The above is as stated in this application. Figure 1The method executed by the target detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0121] The electronic device can also perform Figure 1 The method executed by the target detection device, and the implementation of the target detection device in... Figure 1 The functions of the embodiments shown are not described in detail here.
[0122] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the target detection device in the illustrated embodiment is specifically used to perform:
[0123] Acquire fisheye images captured by a fisheye camera;
[0124] Based on the target detection model, the target detection box of the target in the fisheye image is obtained;
[0125] The position offset of the target is determined based on the first position point in the fisheye image and the second position point in the target detection box;
[0126] The second position point in the target detection box is corrected based on the position offset of the target to obtain the third position point in the target detection box, and the third position point is used as the positioning result of the target in the fisheye image.
[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0128] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0136] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.
Claims
1. A target detection method, the method comprising: Acquire fisheye images captured by a fisheye camera; Based on the target detection model, the target detection box of the target in the fisheye image is obtained; The position offset of the target is determined based on the first position point in the fisheye image and the second position point in the target detection box; The second position point in the target detection box is corrected according to the position offset of the target to obtain the third position point in the target detection box, and the third position point is used as the positioning result of the target in the fisheye image; The position offset includes at least the offset amount, offset direction, and correction offset; Let the center point of the fisheye image be (Ix, Iy); the center point of the target detection box be (bx, by), and the width and height of the target detection box be (w, h); then the position offset of the target detection box relative to the center of the image is... dx = abs(bx - Ix) dy = abs(by - Iy) Offset direction: fx = -dx / abs(dx) fy = -dy / abs(dy) Correcting bias: delta_x = (w / 2)*(dx / Ix)^sigma delta_y=(h / 2)*(dy / Iy)^sigma Wherein, sigma is called the adjustment coefficient. The final corrected coordinates of the third position point are: ox = bx + delta_x * fx oy = by + delta_y * fy.
2. The method of claim 1, wherein, The step of correcting the second position point in the target detection box based on the position offset of the target to obtain the third position point in the target detection box, and using the third position point as the positioning result of the target in the fisheye image, includes: The second location point in the target detection box is used as the initial positioning result of the target in the fisheye image; The initial positioning result in the fisheye image is corrected based on the position offset of the target to obtain the final positioning result of the target in the fisheye image, wherein the final positioning result of the target in the fisheye image is the third position point re-determined in the target detection box.
3. The method of claim 2, wherein, The first location point includes the center point of the fisheye image, and the second location point includes the center point of the target detection box.
4. The method of claim 3, wherein, The step of determining the position offset of the target based on the first position point in the fisheye image and the second position point in the target detection box includes: The relative position of the target in the fisheye image is determined based on the center point of the fisheye image and the center point of the target detection box. Based on the relative positional relationship, the positional offset of the target is determined.
5. The method of claim 1, wherein, The acquisition of images captured by the fisheye camera includes: The Y-axis of the imaging plane of the fisheye camera is parallel to the road surface. The image is then subjected to distortion correction to obtain the corrected image.
6. The method of claim 1, wherein, The step of obtaining the target detection bounding box of the target in the image based on the target detection model includes: The object detection model based on convolutional neural networks detects objects in the image and outputs the object detection box.
7. A target detection device, wherein, The device includes: The acquisition module is used to acquire fisheye images captured by the fisheye camera; The detection module is used to obtain the target detection box of the target in the fisheye image according to the target detection model; The determination module is used to determine the position offset of the target based on a first position point in the fisheye image and a second position point in the target detection box; The correction module is used to correct the second position point in the target detection box according to the position offset of the target, obtain the third position point in the target detection box, and use the third position point as the positioning result of the target in the fisheye image; The position offset includes at least the offset amount, offset direction, and correction offset; Let the center point of the fisheye image be (Ix, Iy); the center point of the target detection box be (bx, by), and the width and height of the target detection box be (w, h); then the position offset of the target detection box relative to the center of the image is... dx = abs(bx - Ix) dy = abs(by - Iy) Offset direction: fx = -dx / abs(dx) fy = -dy / abs(dy) Correcting bias: delta_x = (w / 2)*(dx / Ix)^sigma delta_y=(h / 2)*(dy / Iy)^sigma Wherein, sigma is called the adjustment coefficient. The final corrected coordinates of the third position point are: ox = bx + delta_x * fx oy = by + delta_y * fy.
8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 6.
9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 6.