Method, device, electronic device and storage medium for detecting target object
Through the multi-camera object detection system, the problem of target objects being blocked in video surveillance is solved by using image fusion technology, improving the accuracy of detection, and is suitable for crowd density analysis and pedestrian tracking.
Patent Information
- Application Number
- CN202011359126.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-27
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-11-27
AI Technical Summary
In video surveillance, especially in crowd density analysis scenarios, the problem of target objects being blocked leads to inaccurate target detection, especially in pedestrian-intensive areas, which is difficult for a single camera to accurately detect all pedestrians, affecting pedestrian tracking and crowd density detection.
A multi-camera object detection system is adopted to obtain images of different shooting angles, map them to preset planes and perform image fusion, generate a fused image for object detection, thereby identifying the obstructed object.
The accuracy of target detection is improved, the obscured target objects can be identified in the fusion image, and the accuracy of pedestrian and population density analysis is enhanced.
Smart Images

Figure CN112287901B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of video surveillance technology, and in particular, relates to a method, device, electronic device and storage medium for detecting a target object. Background Art
[0002] With the widespread application of video surveillance, the demand for video surveillance functions is no longer limited to video playback and tracing. More and more application scenarios, such as crowd density analysis, require target detection of monitored objects, such as pedestrians, within the video surveillance area.
[0003] However, when there are many monitored objects, the monitored objects often block each other, or the monitored objects are blocked by other objects, resulting in the problem of inaccurate target detection of the monitored objects. Summary of the invention
[0004] The embodiments of the present application provide a method, device, electronic device and storage medium for detecting a target object, which can solve the problem of the detected object being blocked and improve the accuracy of target detection.
[0005] In a first aspect, an embodiment of the present application provides a method for detecting an object, comprising:
[0006] Acquire a plurality of first images of the target area at the same time; at least two of the plurality of first images are shot at different angles;
[0007] According to the first mapping relationship corresponding to each of the first images, each of the first images is mapped to a preset plane to obtain a plurality of second images;
[0008] Performing an image fusion operation on the multiple second images to obtain a fused image;
[0009] Target detection is performed based on the fused image.
[0010] It should be understood that by acquiring first images taken at at least two different shooting angles, mapping the first images of multiple angles for the target area to a preset plane, and fusing the images mapped to the preset plane, both the occluded targets and the unoccluded targets can be integrated into one fused image, and target detection can be performed based on the fused image, so that the occluded targets can be identified, thereby improving the accuracy of target detection.
[0011] In a second aspect, an embodiment of the present application provides a target object detection device, including:
[0012] A first image acquisition module, used to acquire a plurality of first images of the target area at the same time; at least two of the plurality of first images have different shooting angles;
[0013] A second image acquisition module, configured to map each of the first images to a preset plane according to a first mapping relationship corresponding to each of the first images, so as to obtain a plurality of second images;
[0014] A fused image acquisition module, used for performing an image fusion operation on all second images to obtain a fused image;
[0015] The object detection module performs object detection based on the fused image.
[0016] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0017] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method steps described in the first aspect are implemented.
[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including: the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in the first aspect are implemented.
[0019] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method steps described in the first aspect above.
[0020] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0022] Figure 1 is a schematic diagram of a multi-camera object detection system provided by an embodiment of the present application;
[0023] Figure 2 It is a flowchart of a method for detecting a target object provided in an embodiment of the present application;
[0024] Figure 3 is a schematic diagram of a first angle image provided by an embodiment of the present application;
[0025] Figure 4is a schematic diagram of a second angle image provided by an embodiment of the present application;
[0026] Figure 5 is a schematic diagram of image fusion provided by an embodiment of the present application;
[0027] Figure 6 This is a schematic diagram of single-camera head detection provided by an embodiment of the present application;
[0028] Figure 7 is a pedestrian detection position map provided by an embodiment of the present application;
[0029] Figure 8 It is a schematic diagram of the structure of a device for detecting a target object provided in an embodiment of the present application;
[0030] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0032] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0033] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0034] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0035] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0036] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0037] With the widespread application of video surveillance, the demand for video surveillance functions is no longer limited to video playback and tracing. More and more application scenarios, such as crowd density analysis, require target detection of monitored objects, such as pedestrians, within the video surveillance area.
[0038] However, when there are many monitored objects, the monitored objects often block each other, or are blocked by other objects, resulting in the problem of inaccurate target detection of the monitored objects.
[0039] Taking pedestrians as an example, in areas with high pedestrian density, mutual occlusion between pedestrians makes it difficult for a single camera to accurately detect all pedestrians in the target area, thus creating obstacles for subsequent applications such as pedestrian tracking and crowd density detection.
[0040] In order to solve the above problems, an embodiment of the present application provides a multi-camera object detection system and an object detection method applied to the detection system. Figure 1 The figure shows a multi-camera object detection system 100 provided by an embodiment of the present application, which includes a plurality of cameras 110 for a target area (or a monitoring area) 120. The plurality of different cameras 110 take pictures of the target area from different angles. The number of cameras 110 is greater than or equal to 2. For the purpose of illustration and not limitation, Figure 1 4 cameras 110 are shown in FIG. The target area at least includes the overlapping area of the fields of view of the multiple cameras.
[0041] In some embodiments, each camera communicates with an electronic device provided in an embodiment of the present application, and the electronic device is used to implement the target detection method provided in the implementation of the present application. The electronic device is also used to include but is not limited to processing images captured by each camera, target detection, target tracking, etc. The communication method between the electronic device and each camera includes but is not limited to wired communication or wireless communication. In some embodiments, the electronic device can be a cloud computing device.
[0042] An embodiment of the present application provides a method for detecting a target object based on multi-image information fusion, which can be implemented by the above-mentioned electronic device. The electronic device obtains the original image taken by multiple cameras of the target area, or obtains a feature map obtained by extracting features from the original image, or obtains a target object marked image obtained by marking the original image. The original image, feature map or target object marked image is mapped to a preset plane to obtain a fused image. By performing target detection based on the fused image, the obscured target object can be identified, thereby improving the accuracy of target detection. The detected targets include but are not limited to pedestrians, faces, heads and other objects. In some embodiments, the preset plane is a ground plane.
[0043] Figure 2 The method for detecting a target object provided in the embodiment of the present application is shown, and is applied to the above Figure 1 The multi-camera object detection system shown in the figure is implemented by software and / or hardware of an electronic device of the multi-camera object detection system in some embodiments. Figure 2 As shown, the method includes steps S110 to S140. The specific implementation principle of each step is as follows:
[0044] S110, acquiring a plurality of first images of a target area at the same time; at least two of the plurality of first images have different shooting angles.
[0045] In some embodiments, the first image is an original image taken by cameras at various shooting angles in a multi-camera object detection system, and the electronic device determines whether the image is taken at the same time through a timestamp. The target area includes the overlapping area of the images of the cameras at various shooting angles.
[0046] In some embodiments, the first image is a feature map of the original image. Acquiring multiple first images of the target area at the same time includes: acquiring original images taken by multiple cameras for the target area, the original images having the same timestamp; extracting feature maps of each of the original images; and using each of the feature maps as the first image. In a specific example, the original image can be feature extracted by a convolutional neural network (CNN) to obtain a feature map. In some examples, the feature map obtained by the CNN network can also be upsampled to obtain a feature map of the same size as the original image, so as to facilitate the operation of mapping to a preset plane.
[0047] In some embodiments, the first image is a target object labeled image, that is, an image in which the target object is marked in the image. It should be understood that the target object labeled image can be an image that only includes the marked target object position and classification, or it can be an image in which the target object position is marked on the original image. In a non-limiting manner, a rectangular frame is used to mark the target object. Acquiring multiple first images of the target area at the same time, including: acquiring original images taken by multiple cameras for the target area, the original images having the same timestamp; performing target object detection on each of the original images to obtain a target object labeled image; using each of the target object labeled images as the first image. Detecting the target object in the original image includes but is not limited to using a region (Region-CNN, R-CNN), a pyramid pooling network (Spatial PyramidPooling Networks, SPPNet), Fast R-CNN, Faster R-CNN, YOLO, (Single Shot MultiBoxDetector, SSD) and other networks to mark the target object in a single original image.
[0048] S120: Map each of the first images to a preset plane according to a first mapping relationship corresponding to each of the first images to obtain a plurality of second images.
[0049] In some embodiments, the preset plane may be a ground plane, that is, a plane where the ground of the space where each camera is arranged is located. Using the ground plane as the preset plane does not require the calculation of complex occlusion weights and transformation reference planes, and the implementation is simpler. When viewing pedestrians on the ground from a vertical overlooking angle, if the monitoring scene is not large, it can be considered that what is seen is the top of the head, and the mutual occlusion of targets will be greatly improved.
[0050] In some embodiments, each first image may be an original image of a camera or an image after processing the original image. In some embodiments, before acquiring multiple first images of the target area at the same time, it also includes: acquiring the homography matrix of each of the first images mapped to a preset plane; using the homography matrix of each of the first images as the first mapping relationship corresponding to the first image. In a specific example, a checkerboard placed on a preset plane is provided, and first angle images taken by multiple cameras at respective first preset angles are acquired, wherein the first angle image includes the checkerboard and the target area; second angle images taken by each camera at respective second preset angles are acquired; the first angle image includes the checkerboard; for the first angle image and the second angle image of each camera, the homography matrix is obtained according to the first position parameters of each reference point of the checkerboard in the first angle image and the second position parameters in the second angle image. A homography matrix can be solved using at least four feature points in the two images, and then this homography matrix can be used to transform a coordinate in image 1 to a corresponding position in image 2. As Figure 3 The first angle image schematic diagram provided by an embodiment of the present application is shown, wherein the first preset angle is the angle used by the camera when tracking the target object; Figure 4 The second angle image schematic diagram provided by an embodiment of the present application is shown, and the second preset angle is the angle adopted by the camera to look down and shoot the chessboard. In a more specific implementation example, by placing a chessboard image on the ground plane, and then obtaining the coordinates of the four corner points on the chessboard image on the ground plane (0,0), (widht-1,0), (0,height-1), (wdith-1,height-1), widht indicates the width of the chessboard, and height indicates the height of the chessboard. At the same time, the corner points are extracted on the plane of the original image shot, and the coordinate values of the corner points corresponding to the four points on the ground plane in the image space are obtained. Through the correspondence between the coordinate points of the four corner points on the ground plane and the image plane, based on the getPerspectiveTransform() function of OpenCV, the homography matrix H of the projection transformation relationship from the image plane to the ground plane is obtained. i , i identifies the corresponding camera number.
[0051] In some embodiments, before acquiring multiple first images of the target area at the same time, the method further includes: calibrating each camera using a chessboard, and calculating the intrinsic parameters and distortion parameter matrix of each camera. A common method is the Zhang Zhengyou plane calibration method, which prints a chessboard template and sticks it on a plane. The width, height, and number of row and column corner points of the chessboard are known, and several template images are taken from different angles to detect feature points (Harris corner points) in the image. The calibration function calibrateCamera() provided by OpenCV is used for calibration calculation.
[0052] S130: Perform an image fusion operation on the multiple second images to obtain a fused image.
[0053] In one embodiment, performing an image fusion operation on the multiple second images includes superimposing pixels of the multiple second images. Figure 5 is a schematic diagram of image fusion provided by an embodiment of the present application. Figure 5 As shown, the images captured by the cameras at various angles are mapped to the preset plane and then fused into a fused image by pixel superposition. Figure 5 The target object to be detected is not shown.
[0054] In some embodiments, the first image is a feature map, and after the first image is mapped to a preset plane, the features of each feature map are still maintained. In some embodiments, each feature of the second image is fused to be superimposed on the preset plane.
[0055] In some embodiments, the first image is a target object marking image, and the fused image is a superposition of the markings. Figure 6 : is a schematic diagram of single camera head detection provided by an embodiment of the present application. For an original image captured by a camera, target detection is performed on the original image. Figure 6 The detection results of human heads in some embodiments are shown. The single image object detection result is extracted to obtain an image containing the object mark, and the image is used as the first image. The original images taken by the camera at each shooting angle are processed in the same way to obtain multiple first images. These first images are mapped to a preset plane and the marks are superimposed to obtain the following: Figure 7 The pedestrian detection position diagram shown in FIG7 , the dot in the image represents a detected pedestrian.
[0056] S140: Detect an object based on the fused image.
[0057] In some embodiments, the first image is an original image captured by cameras at various shooting angles in a multi-camera object detection system, and the fused image is a fusion of multiple original images mapped to a preset plane. Exemplarily, the above-mentioned deep learning neural network model for object detection is used to perform object detection on the first fused image. The detected object is annotated with a labeling box to obtain an object detection result.
[0058] In some embodiments, the first image is a feature map of the original image. After mapping multiple feature maps to a preset plane, a fusion process is performed to obtain a fused image containing features of multiple feature maps. The fused image is clustered to obtain a clustering result of similar features. In a specific example, the features in the feature map are pedestrian head features. After clustering the head features, a target detection result of the position distribution of the pedestrian is obtained.
[0059] In some embodiments, the first image is a target object labeled image, and the fused image is clustered to obtain, that is, the same or similar labels are processed as one target object to obtain a target object detection result.
[0060] It should be understood that by acquiring first images taken at at least two different shooting angles, mapping the first images of multiple angles for the target area to a preset plane, and fusing the images mapped to the preset plane, both the occluded targets and the unoccluded targets can be integrated into one fused image, and target detection can be performed based on the fused image, so that the occluded targets can be identified, thereby improving the accuracy of target detection.
[0061] In the above Figure 2 Based on the embodiment of the method for object detection shown in FIG. 1 , after object detection is performed based on the fused image, steps S151 to S154 are further included:
[0062] S151 , according to the first position of each detected target object in the fused image, inversely map and transform the second position of each target object in the first image.
[0063] It can be seen from the above embodiments that the fused image is a plurality of first images mapped to a second image through a preset mapping relationship. Therefore, the first mapping relationship can be used to inversely map and transform the second position of each target object in the first image according to the first position of each detected target object in the fused image.
[0064] S152: Mark each of the target objects in each of the first images according to the second position to obtain a third image.
[0065] In some embodiments, each detected target object may be marked in the first image using a rectangular frame, and the marked first image is used as the third image.
[0066] S153, for each of the marked targets, according to the screening parameters, screening the partial images containing the target in each of the third images, and extracting the partial images containing the target.
[0067] S154: Store the local image.
[0068] In some embodiments, for each of the labeled targets, partial images in each of the third images containing the target are screened to obtain an optimal partial image, in which the target is contained.
[0069] In some embodiments, each detected target object is marked in the first image using a rectangular frame. The local image may be an image within each rectangular frame.
[0070] The screening parameters include at least one of the following: the angle of the target object in the partial image; the clarity of the target object in the partial image; the size of the annotation box of the target object in the partial image; and the brightness of the partial image.
[0071] The angle of the target object in the partial image is a preset angle. In a specific example, the target object is a human head, and the angle of the target object is the front of the human head. Exemplarily, for the same target object, a partial image of the front of the face is selected and the partial image is extracted.
[0072] The clarity of the image can be detected by using entropy function method, gradient function method, variance function method, deep learning method, etc. Exemplarily, for the same target object, the clearest local image of the target object is selected from multiple third images, and the local image is extracted.
[0073] Among them, in some embodiments, the target object is detected by the above-mentioned deep learning neural network model for target detection. While the deep learning network marks the object of the target object, it also marks the probability value of the object in the marking box being the target object. The larger the probability value, the clearer the target object is. Exemplarily, for the same target object, a local image with the highest clarity of the target object is selected and the local image is extracted.
[0074] In some embodiments, for the same target object, a local image with the largest rectangular frame area marking the target object is selected and the local image is extracted.
[0075] In some embodiments, for the same target object, a local image with the maximum brightness of the target object or a brightness within a preset threshold range is selected and the local image is extracted.
[0076] When the embodiment of the present application is actually implemented, one or more of the above screening parameters can be selected according to actual conditions to screen the local image in the third image. In a specific example, multiple screening parameters are selected from the above screening parameters, each screening parameter has a weight, and the third image with the largest weighted sum of the screening parameters is used as the fourth image.
[0077] It should be understood that by acquiring images of the target area taken by cameras from multiple angles at the same time, and filtering out the best local images through the above steps, and saving the local images, the best images of all targets that need to be detected in the target area at that moment can be obtained, thereby solving the problem of occlusion of the detection object, improving the accuracy of target detection, and saving storage space.
[0078] In the above Figure 2 Based on the embodiment of the object detection method shown, after the object detection is performed based on the fused image, step S161 is further included:
[0079] S161: Calculate the actual distance between each detected target object according to the second preset mapping relationship.
[0080] In some embodiments, the purpose of detecting the target objects is to monitor the distance between the target objects. For example, during an epidemic, it is necessary to detect the distance between pedestrians and control traffic flow or remind pedestrians in a timely manner.
[0081] refer to Figure 7 During the epidemic, in some special occasions (public places, such as queuing scenes), the target position information can be used to control the distance between people. In the process of using the chessboard to calibrate the camera, the actual width of the chessboard is known to be w 实 and the pixel distance w of the chessboard in ground plane coordinates 像 , it is known that the pixel distance between the two targets is d 像 , the actual distance can be estimated: d 实 =w 实 *d 像 / w 像 , the calculation formula can be used as the second mapping relationship to calculate the actual distance between the two targets.
[0082] In some embodiments, when there are multiple targets in the target area, after calculating the actual distance between each detected target, the density of the targets in the target area can be analyzed. If the density of the targets is greater than a preset value, an early warning or other operations are performed.
[0083] Corresponding to the above Figure 2 The method of object detection shown in FIG. Figure 8The present invention shows a device M100 for detecting a target object provided in an embodiment of the present application, including:
[0084] The first image acquisition module M110 is used to acquire a plurality of first images of the target area at the same time; at least two of the plurality of first images are shot at different angles;
[0085] A second image acquisition module M120, configured to map each of the first images to a preset plane according to a first mapping relationship corresponding to each of the first images, to obtain a plurality of second images;
[0086] A fused image acquisition module M130 is used to perform an image fusion operation on all second images to obtain a fused image;
[0087] The object detection module M140 performs object detection based on the fused image.
[0088] It can be understood that the various implementation modes and implementation combinations in the above embodiments and their beneficial effects are also applicable to this embodiment and will not be described in detail here.
[0089] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig. 9 As shown, the electronic device D10 of this embodiment includes: at least one processor D100 ( Fig. 9 Only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100 are shown, and when the processor D100 executes the computer program D102, the steps in any of the above-mentioned method embodiments are implemented. Alternatively, when the processor D100 executes the computer program D102, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0090] The electronic device D10 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will appreciate that Fig. 9 This is only an example of the electronic device D10 and does not constitute a limitation on the electronic device D10. The electronic device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.
[0091] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0092] In some embodiments, the memory D101 may be an internal storage unit of the electronic device D10, such as a hard disk or memory of the electronic device D10. In other embodiments, the memory D101 may also be an external storage device of the electronic device D10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device D10. Further, the memory D101 may also include both an internal storage unit of the electronic device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.
[0093] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0094] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0095] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0096] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0097] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0099] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0103] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for detecting a target object, characterized in that: include: Acquire multiple first images of the target area at the same time; At least two of the plurality of first images are shot at different angles; According to the first mapping relationship corresponding to each of the first images, each of the first images is mapped to a preset plane to obtain a plurality of second images; Performing an image fusion operation on the multiple second images to obtain a fused image; Performing target object detection based on the fused image; Acquiring multiple first images of the target area at the same time, including: Acquire original images captured by multiple cameras for the target area, wherein the original images have the same timestamp; Extracting a feature map of each of the original images; Upsampling the feature map to obtain a feature map of the same size as the original image; Taking each of the feature maps having the same size as the original image as the first image; After detecting the target object based on the fused image, the method further includes: According to the first position of each detected target object in the fused image, inverse mapping transforms the second position of each target object in the first image; According to the second position, marking each of the targets in each of the first images to obtain a third image; For each of the marked targets, according to the screening parameters, the partial images containing the target in each of the third images are screened, and the partial images containing the target are extracted; storing the partial image; The screening parameters include at least one of the following: The angle of the target object in the partial image; the clarity of the target object in the partial image; the size of the annotation box of the target object in the partial image; and the brightness of the partial image.
2. The method according to claim 1, characterized in that Before acquiring a plurality of first images of the target area at the same time, the method further includes: Obtaining a homography matrix of each of the first images mapped to a preset plane; The homography matrix of each of the first images is used as the first mapping relationship corresponding to the first image.
3. The method according to claim 1, characterized in that Acquiring multiple first images of the target area at the same time, including: Acquire original images captured by multiple cameras for the target area, wherein the original images have the same timestamp; Performing target object detection on each of the original images to obtain a target object marked image; Each of the target object marked images is used as the first image.
4. The method according to any one of claims 1 to 3, characterized in that After detecting the target object based on the fused image, the method further includes: According to the second preset mapping relationship, the actual distance between each detected target object is calculated.
5. A device for detecting an object, characterized in that: include: A first image acquisition module, used to acquire multiple first images of the target area at the same time; At least two of the plurality of first images are shot at different angles; A second image acquisition module, configured to map each of the first images to a preset plane according to a first mapping relationship corresponding to each of the first images, so as to obtain a plurality of second images; A fused image acquisition module, used for performing an image fusion operation on all second images to obtain a fused image; An object detection module, performing object detection based on the fused image; Acquiring multiple first images of the target area at the same time, including: Acquire original images captured by multiple cameras for the target area, wherein the original images have the same timestamp; Extracting a feature map of each of the original images; Upsampling the feature map to obtain a feature map of the same size as the original image; Taking each of the feature maps having the same size as the original image as the first image; After detecting the target object based on the fused image, the method further includes: According to the first position of each detected target object in the fused image, inverse mapping transforms the second position of each target object in the first image; According to the second position, marking each of the targets in each of the first images to obtain a third image; For each of the marked targets, according to the screening parameters, the partial images containing the target in each of the third images are screened, and the partial images containing the target are extracted; storing the partial image; The screening parameters include at least one of the following: The angle of the target object in the partial image; the clarity of the target object in the partial image; the size of the annotation box of the target object in the partial image; and the brightness of the partial image.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.
7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
