A multi-threshold target detection method based on multiple projections

Abstract

The present invention provides a multi-threshold target detection method based on multiple projections, comprising the following steps: S1, constructing multiple projection surfaces at multiple distance scales relative to a camera, and projecting the camera's imaging onto a corresponding projection surface one based on the camera's internal and external parameters to obtain an image; S2, performing target recognition with a low confidence threshold requirement on the image, and obtaining the area range of the target in the detection image, determining the key points of the area range, and determining the pixel coordinates of the key points of the area range in the original image; the present invention can eliminate panoramic-related blind spots. If a pedestrian is just near a stitching line of the current panoramic view, the pedestrian is avoided by dynamically adjusting stitching lines at different angles to prevent the pedestrian from appearing just on or near the stitching line, thereby eliminating the panoramic imaging pedestrian blind spot and solving some safety hazards caused by the panoramic blind spot during driving.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-threshold target detection method based on multi-projection. Background Art

[0002] The existing technology usually scales the original image of the target camera or the area where target recognition is required by a certain ratio, and then directly inputs it into the neural network for detection.

[0003] The Chinese Public Invention (patent application number CN112712037A) discloses an on-vehicle environment perception method and system based on panoramic images and target detection, which splices environmental images into a panoramic image and captures detailed images of the target based on the detected target position, which can effectively detect the target, facilitate the driver to obtain a complete and intuitive understanding of the surrounding environment, and effectively improve the driver's efficiency in observing the environment. Automatically tracking targets in the environment, issuing prompt sounds and displaying detailed target images can display detailed information of environmental targets to the driver as quickly as possible, reducing the risk of the driver missing surrounding targets. However, there are still certain problems:

[0004] First, it solves the problem that only the image near the vehicle body is clear, and the image in the distance is distorted and the target cannot be effectively detected;

[0005] Second, after scaling, the features of smaller targets are completely lost and become unrecognizable;

[0006] 3. If scaling is not performed, or the scaling is not sufficient, the area that needs to be identified may be very large, which will consume a huge amount of computing power for AI reasoning calculations; on platforms with limited computing power, the practicality is relatively poor;

[0007] 4. Usually, in real physical space, users are not interested in all the information in the intercepted area defined in the image and its corresponding viewing cone. Simply intercepting the original image for detection will cause a huge waste of computation and may introduce AI detection misjudgment. For example, when detecting pedestrians while the vehicle is driving, we only need to detect targets standing on the ground or close to the ground, such as Figure 1 Targets in the range area shown;

[0008] Therefore, a multi-threshold target detection method based on multi-projection is proposed. Summary of the invention

[0009] In view of this, an embodiment of the present invention hopes to provide a multi-threshold target detection method based on multiple projections to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0010] The technical solution of the embodiment of the present invention is implemented as follows: a multi-threshold target detection method based on multiple projections, comprising the following steps:

[0011] S1, constructing multiple projection surfaces at multiple distance scales relative to the camera, and projecting the camera's imaging onto corresponding projection surface one based on the camera's internal and external parameters to obtain an image;

[0012] S2, performing target recognition with a low confidence threshold requirement on the image, and obtaining the region range of the target in the detection image, determining the key points of the region range, and determining the pixel coordinates of the key points of the region range in the original image;

[0013] S3, based on the camera internal and external parameters and the pixel coordinates of the key points in the area range in the original image, reversely obtain the actual position coordinates of the target in the geodetic coordinate system, and filter and identify the target;

[0014] S4, based on the actual position coordinates of the target in the geodetic coordinate system, construct a vertical axis perpendicular to the ground center axis, and take this axis as the center and perpendicular to the direction of the camera connection to construct a new projection plane 2, magnify the image of the target object in the projection plane 2, and adjust it to a state close to a fully normal view;

[0015] S5, projecting the camera image onto projection surface 2 according to the internal and external parameters of the camera;

[0016] S6. On the second projection surface after completing the above steps, perform deep learning detection and set a higher detection confidence threshold to obtain the area range of the target in the detection image, determine the key points of the area range, and find their pixel coordinates in the original image;

[0017] S7, again using the internal and external parameters of the camera and the pixel coordinates of the key points of the region in the original image, reversely obtain the actual position coordinate 2 of the object in the geodetic coordinate system;

[0018] S8, comparing the geodetic coordinate position coordinate 1 and the geodetic coordinate position coordinate 2, and filtering if the position offset is greater than a certain threshold;

[0019] S9. After all the above steps, the targets still detected and the related location areas are determined as the required target detection results.

[0020] In some embodiments: the area range key points include the bottom midpoint and the top midpoint of the area range.

[0021] In some embodiments: in the step S1, after projecting the image of the camera onto a corresponding projection surface based on the internal and external parameters of the camera, the step further includes performing histogram equalization on the image.

[0022] In some embodiments: in step S3, when filtering and identifying targets, the following steps are included:

[0023] S31, calculating the actual height of the target according to the pixel coordinates of the top of the detection area and the position coordinates of the geodetic coordinate system obtained in S4;

[0024] S32, filtering according to the calculated position coordinates, filtering out targets outside the area of ​​interest;

[0025] S33. Filtering is performed according to the calculated height of the target to filter out the identified targets that obviously exceed the reasonable height of the target.

[0026] In some embodiments: in step S2, if the confidence level of the detected target is higher than a certain higher threshold, the target detection is determined to be successful, and step S9 is executed to output corresponding data.

[0027] In some embodiments: in step S2, if the confidence level of the detected target is lower than a certain higher threshold, step S4 is executed.

[0028] In some embodiments: in the step S5, when the camera image is projected onto the second projection surface according to the internal and external parameters of the camera, the step also includes performing histogram equalization on the image.

[0029] In order to solve the above technical problems, another technical solution adopted by the present application is: a multi-threshold target detection device based on multi-projection, comprising:

[0030] A construction module is used to construct multiple projection surfaces and project the camera images onto corresponding projection surfaces according to the internal and external parameters of the camera;

[0031] A processing module, used for performing histogram equalization on the image projected onto the projection surface;

[0032] The recognition module is used to identify the target with low confidence threshold requirements in the image, obtain the area range of the target in the detection image, determine the key points of the area range, and determine its pixel coordinates in the original image;

[0033] An acquisition module is used to reversely acquire the actual position coordinates of the target in the geodetic coordinate system;

[0034] Filtering module, used to filter targets that do not meet the requirements;

[0035] A comparison module, used for comparing the geodetic coordinate position coordinate one and the geodetic coordinate position coordinate two;

[0036] Output module, used to output target detection results.

[0037] To solve the above technical problems, another technical solution adopted in the present application is: a computer device, which includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of the multi-threshold target detection method based on multiple projections as described above.

[0038] In order to solve the above technical problems, another technical solution adopted by the present application is: a storage medium storing program instructions capable of implementing the multi-threshold target detection method based on multiple projections as described above.

[0039] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:

[0040] The present invention can eliminate panoramic-related blind spots. If a pedestrian happens to be near the stitching line of the current panoramic view, the pedestrian can be avoided by dynamically adjusting stitching lines at different angles to prevent the pedestrian from appearing exactly on or near the stitching line, thereby eliminating the panoramic imaging pedestrian blind spot and solving some safety hazards caused by the panoramic blind spot during driving.

[0041] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of the area of ​​interest to the user;

[0044] Figure 2 It is a flowchart of a multi-threshold target detection method based on multiple projections of the present invention;

[0045] Figure 3 It is a schematic diagram of the functional modules of the multi-threshold target detection system based on multiple projections of the present invention;

[0046] Figure 4 A schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0047] Figure 5 Schematic diagram of the structure of a storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The terms "first", "second", "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0050] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] Figure 2 is a flowchart of a multi-threshold target detection method based on multiple projections according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not based on Figure 1 The process sequence shown is limited. Figure 2 It includes: a multi-threshold target detection method based on multiple projections, comprising the following steps:

[0052] S1. Construct multiple projection surfaces at multiple distance scales relative to the camera, and project the camera's image onto the corresponding projection surface according to the camera's internal and external parameters;

[0053] S2, performing histogram equalization;

[0054] S3. Then, the image is subjected to target recognition with a low confidence threshold requirement, and the target region in the detection image is obtained, the key points of the region are determined (such as the bottom midpoint and the top midpoint of the region), and their pixel coordinates in the original image are found.

[0055] S4, using the internal and external parameters of the camera and the pixel coordinates of the bottom midpoint of the detection area in the original image, reversely obtain the actual position coordinates of the object in the geodetic coordinate system;

[0056] S5, and calculating the actual height of the target according to the pixel coordinates of the top of the detection area and the position coordinates of the geodetic coordinate system obtained in S4;

[0057] S6, filtering according to the position coordinates calculated in S4, filtering out the targets outside the area of ​​interest;

[0058] S7, filtering according to the height of the target calculated in S5, filtering out the identified targets that obviously exceed the reasonable height of the target;

[0059] S8: At this time, if the confidence of the target detected by S3 is higher than a certain higher threshold, we determine that the target detection is successful and output the corresponding data; if it is lower than the threshold, execute S9;

[0060] S9, if S4 constructs a projection plane perpendicular to the ground axis based on the geodetic coordinates of the target obtained by S4, with the center being the ground axis and perpendicular to the direction of the camera connection, a new slightly smaller projection plane (capable of enclosing the identified target with a margin) is constructed, the target object will be enlarged in the projection plane and adjusted to a nearly fully normal view state;

[0061] S10, projecting the camera image onto the projection surface constructed in S9 according to the internal and external parameters of the camera;

[0062] S11, performing histogram equalization again on the new projection surface;

[0063] S12. On the projection surface where the above steps are completed, deep learning detection is performed again, and a higher detection confidence threshold is set. The area range of the target in the detection image is obtained, the key points of the area range are determined (such as the bottom midpoint and the top midpoint of the area range), and their pixel coordinates in the original image are found.

[0064] S13, using the internal and external parameters of the camera and the pixel coordinates of the bottom midpoint of the detection area in the original image, reversely obtain the actual position coordinates of the object in the geodetic coordinate system

[0065] S14. Compare the geodetic coordinates obtained for the first time and the second time, and if the position offset is greater than a certain threshold, filter it.

[0066] S15. After all the above steps, the detected target and the related location area are the target detection results we need.

[0067] Figure 3 is a schematic diagram of functional modules of a multi-threshold target detection system based on multiple projections according to an embodiment of the present application. Figure 3 As shown, a multi-threshold target detection system based on multiple projections includes:

[0068] A construction module is used to construct multiple projection surfaces and project the camera images onto corresponding projection surfaces according to the internal and external parameters of the camera;

[0069] A processing module, used for performing histogram equalization on the image projected onto the projection surface;

[0070] The recognition module is used to identify the target with low confidence threshold requirements in the image, obtain the area range of the target in the detection image, determine the key points of the area range, and determine its pixel coordinates in the original image;

[0071] An acquisition module is used to reversely acquire the actual position coordinates of the target in the geodetic coordinate system;

[0072] Filtering module, used to filter targets that do not meet the requirements;

[0073] A comparison module, used for comparing the geodetic coordinate position coordinate one and the geodetic coordinate position coordinate two;

[0074] Output module, used to output target detection results.

[0075] For other details about the technical solutions for implementing each module in the multi-threshold target detection system based on multiple projections in the above-mentioned embodiment, please refer to the description of the multi-threshold target detection method based on multiple projections in the above-mentioned embodiment, which will not be repeated here.

[0076] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0077] See also Figure 4 , Figure 4 Schematic diagram of the structure of the computer device of the embodiment of the present application. Figure 4As shown, the computer device 30 includes a processor 31 and a memory 32 coupled to the processor 31 .

[0078] The memory 32 stores program instructions. When the program instructions are executed by the processor 31, the processor 31 executes the steps of the multi-threshold target detection method based on multi-projection in the above embodiment.

[0079] The processor 31 may also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip having signal processing capabilities. The processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0080] See also Figure 5 , Figure 5 Schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores program instructions 41 that can implement all the above methods, wherein the program instructions 41 can be stored in the above storage medium in the form of a software product, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code, or computer devices such as computers, servers, mobile phones, tablets, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0081] When the present invention is working, it can eliminate the panoramic related blind spots. If a pedestrian happens to be near the stitching line of the current panoramic view, the pedestrian can be avoided by dynamically adjusting the stitching lines at different angles to prevent the pedestrian from appearing exactly on the stitching line or near the stitching line, thereby eliminating the panoramic imaging pedestrian blind spot and solving some safety hazards caused by the panoramic blind spot during driving.

[0082] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0083] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A multi-threshold target detection method based on multiple projections, characterized in that: The following steps are involved: S1, constructing multiple projection surfaces at multiple distance scales relative to the camera, and projecting the camera's imaging onto corresponding projection surface one based on the camera's internal and external parameters to obtain an image; S2, performing target recognition with a low confidence threshold requirement on the image, and obtaining the region range of the target in the detection image, determining the key points of the region range, and determining the pixel coordinates of the key points of the region range in the original image; S3, based on the camera internal and external parameters and the pixel coordinates of the key points in the area range in the original image, reversely obtain the actual position coordinates of the target in the geodetic coordinate system, and filter and identify the target; S4, based on the actual position coordinates of the target in the geodetic coordinate system, construct a second projection surface perpendicular to the central axis of the ground, and take this as the center and perpendicular to the direction of the line connecting the camera; S5, projecting the camera image onto projection surface 2 according to the internal and external parameters of the camera; S6. On the second projection surface after completing the above steps, perform deep learning detection and set a higher detection confidence threshold to obtain the area range of the target in the detection image, determine the key points of the area range, and find their pixel coordinates in the original image; S7, again using the internal and external parameters of the camera and the pixel coordinates of the key points of the region in the original image, reversely obtain the actual position coordinate 2 of the object in the geodetic coordinate system; S8, comparing the geodetic coordinate position coordinate 1 and the geodetic coordinate position coordinate 2, and filtering if the position offset is greater than a certain threshold; S9, after all the above steps, the target still detected and the related location area are determined as the required target detection result; In the step S3, when filtering and identifying the target, the following steps are included: S31, calculating the actual height of the target according to the pixel coordinates of the top of the detection area and the position coordinates obtained in the geodetic coordinate system; S32, filtering according to the calculated position coordinates, filtering out targets outside the area of ​​interest; S33. Filtering is performed according to the calculated height of the target to filter out the identified targets that obviously exceed the reasonable height of the target.

2. The multi-threshold target detection method based on multiple projections according to claim 1, characterized in that: The key points of the area range include the bottom midpoint and the top midpoint of the area range.

3. The multi-threshold target detection method based on multiple projections according to claim 1, characterized in that: In the step S1, after projecting the image of the camera onto a corresponding projection surface 1 based on the internal and external parameters of the camera, the step also includes performing histogram equalization on the image.

4. The multi-threshold target detection method based on multi-projection according to claim 1, characterized in that: In step S2, if the confidence level of the detected target is higher than a certain higher threshold, the target detection is determined to be successful, and step S9 is executed to output corresponding data.

5. The multi-threshold target detection method based on multi-projection according to claim 4, characterized in that: In step S2, if the confidence level of the detected target is lower than a certain higher threshold, step S4 is executed.

6. The multi-threshold target detection method based on multi-projection according to claim 4, characterized in that: In the step S5, when the camera image is projected onto the second projection surface according to the internal and external parameters of the camera, the step also includes performing histogram equalization on the image.

7. A device for the multi-threshold target detection method based on multiple projections as claimed in any one of claims 1 to 6, characterized in that: include: A construction module is used to construct multiple projection surfaces and project the camera's imaging onto corresponding projection surfaces according to the camera's internal and external parameters; A processing module, used for performing histogram equalization on the image projected onto the projection surface; The recognition module is used to identify the target with low confidence threshold requirements in the image, obtain the area range of the target in the detection image, determine the key points of the area range, and determine its pixel coordinates in the original image; Position calculation module, used to reversely obtain the actual position coordinates of the target in the geodetic coordinate system; Filtering module, used to filter targets that do not meet the requirements; A comparison module, used for comparing the geodetic coordinate position coordinate one and the geodetic coordinate position coordinate two; Output module, used to output target detection results.

8. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of the multi-threshold target detection method based on multiple projections as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The device stores program instructions capable of implementing the multi-threshold target detection method based on multiple projections as described in any one of claims 1 to 6.

Images