Target detection method and device, computer device and storage medium

Abstract

The application relates to the technical field of computer vision, and discloses a target detection method and device, computer equipment and a storage medium, which comprise the following steps: acquiring images of at least one view angle; performing multiple fusion processes on the images of the same view angle; performing feature extraction on the results obtained by each fusion process as input data of the next fusion process; performing target detection and non-maximum suppression processing on the results of the multiple fusion processes respectively to obtain detection results of each view angle; calculating overlapping areas of the detection results of a target view angle and the detection results of other view angles; and determining the target detection result of the target view angle based on the overlapping detection results and the detection result of the target view angle. The method is not limited to using images of one view angle, and the images are not limited to images at a certain moment, so that the accuracy of target detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular to a target detection method and device, a computer device and a storage medium. BACKGROUND

[0002] With the development of deep learning technology and big data technology, and the improvement of computer computing power, computer vision technology has become one of the most popular advanced technologies in recent years and is widely used in intelligent terminals, mobile Internet applications, finance, telecommunications, automobiles, home, education and other industries. The detection accuracy of targets such as pedestrians, pets or luggage based on computer vision technology is continuously improving, but in actual application environments, due to factors such as light, ground environment, pedestrian clothing, etc., the actual target detection result will be greatly affected.

[0003] In related technologies, a target detection method based on a deep convolutional network is usually directly used to detect a single frame of video image at the current time, but due to various influencing factors, false detection, missed detection and other situations may occur, and the detection accuracy is low. SUMMARY

[0004] Therefore, embodiments of the present application provide a target detection method, device, computer device and storage medium to solve the problem of low target detection accuracy.

[0005] In a first aspect, the embodiments of the present application provide a target detection method, which comprises:

[0006] obtaining images of at least one view of a target scene, the view comprising a target view;

[0007] performing multiple fusion processing on images of the same view, performing feature extraction on the result of each fusion processing based on different scales, and taking the extracted features as input data for the next fusion processing, to obtain multiple fusion processing results;

[0008] performing target detection on the fusion processing results of each view respectively, and performing non-maximum suppression processing on the results of target detection to obtain detection results of each view;

[0009] calculating the overlapping area of the detection results of other views and the detection results of the target view to determine overlapping detection results;

[0010] performing smoothing processing on the detection results of the target view and the overlapping detection results to determine the target detection results of the target view.

[0011] The target detection method provided by the embodiment can obtain images of at least one view angle, perform multiple fusion processes on images of the same view angle, perform feature extraction on results of each fusion process, use the results as input data of the next fusion process, perform target detection and non-maximum suppression on results of the multiple fusion processes respectively, obtain detection results of each view angle, calculate overlapping areas of the detection results of the target view angle and the detection results of other view angles, and determine the target detection result of the target view angle based on the overlapping detection results and the detection result of the target view angle. The method is not limited to using images of one view angle, and the images are not limited to images at one time, and the accuracy of target detection is improved.

[0012] In some embodiments, the obtaining the image of at least one view angle of the target scene comprises:

[0013] The current time image, the background image, at least one image obtained at intervals of a preset first period within a preset time period before the current time, and at least one image obtained at intervals of a preset second period within the preset time period before the current time corresponding to each view angle of the target scene are obtained respectively, the preset second period is less than the preset first period, and the background image does not contain the target.

[0014] In some embodiments, when the multiple fusion processes include three fusion processes, the multiple fusion processes on the images of the same view angle are based on different scales to perform feature extraction on results of each fusion process, and the extracted features are used as input data of the next fusion process to obtain multiple fusion process results, comprising:

[0015] The first target feature, the second target feature, and the third target feature of the target scene under the same view angle are extracted based on different resolutions.

[0016] The first target feature is fused to obtain a first fusion process result.

[0017] The first fusion process result is extracted based on the second target feature to obtain a first extracted feature, and the first extracted feature and the second target feature are fused to obtain a second fusion process result.

[0018] The second fusion process result is extracted based on the third target feature to obtain a second extracted feature, and the second extracted feature and the third target feature are fused to obtain a third fusion process result.

[0019] In some embodiments, the fusion process results of each view angle are detected respectively, and the results of the target detection are subjected to non-maximum suppression to obtain detection results of each view angle, comprising:

[0020] The first fusion processing result, the second fusion processing result and the third fusion processing result corresponding to the same perspective are input into a preset detection head respectively to obtain detection results of different scales.

[0021] The detection results of different scales are subjected to non-maximum suppression processing to determine the detection results of each perspective.

[0022] In some embodiments, the calculation of the overlapping area of the detection results of other perspectives and the detection results of the target perspective to determine the overlapping detection results comprises:

[0023] Converting the detection results of other perspectives into the coordinate system of the detection results of the target perspective;

[0024] Calculating the overlapping area of other perspectives and the target perspective based on the converted detection results to determine the overlapping detection results.

[0025] In some embodiments, the conversion of the detection results of other perspectives into the coordinate system of the detection results of the target perspective comprises:

[0026] Based on the images of other perspectives and the images of the target perspective at the same time, the positions of the cameras corresponding to the images of other perspectives relative to the target camera corresponding to the images of the target perspective are calculated;

[0027] Based on the positions of the cameras corresponding to the images of other perspectives relative to the target camera corresponding to the images of the target perspective and the detection results of other perspectives, the detection results of other perspectives are converted into the coordinate system of the detection results of the target perspective.

[0028] In some embodiments, the smoothing processing of the detection results of the target perspective and the overlapping detection results to determine the target detection results of the target perspective comprises:

[0029] Establishing a space-time channel of each target in the detection results of the target perspective;

[0030] Based on the smoothing processing of the space-time channel according to the overlapping detection results, the false positive results and the false negative results in the detection results are removed to determine the target detection results of the target perspective.

[0031] In a second aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0032] An image acquisition module is configured to acquire images of at least one perspective of a target scene, wherein the perspective comprises a target perspective;

[0033] The fusion processing module is configured to perform multiple fusion processes on the images of the same view, perform feature extraction on the results of each fusion process based on different scales, and use the extracted features as input data for the next fusion process to obtain multiple fusion results.

[0034] The suppression processing module is configured to perform target detection on the fusion results of each view respectively, and perform non-maximum suppression processing on the results of the target detection to obtain detection results of each view.

[0035] The overlap area calculation module is configured to calculate overlap areas of the detection results of other views and the detection results of the target view to determine overlap detection results.

[0036] The smoothing processing module is configured to perform smoothing processing on the detection results of the target view and the overlap detection results to determine target detection results of the target view.

[0037] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, which are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the target detection method in the first aspect or any of the corresponding embodiments.

[0038] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the target detection method in the first aspect or any of the corresponding embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0040] Figure 1 is a flowchart of a target detection method according to some embodiments of the present application;

[0041] Figure 2 is an algorithm flowchart of a target detection method according to some embodiments of the present application;

[0042] Figure 3 is a structural block diagram of a target detection device according to an embodiment of the present application;

[0043] Figure 4 is a hardware structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0045] The target detection method provided by the present application can be applied to indoor or outdoor monitoring scenes. For a fixed scene (for example, a shopping mall entrance), different angle monitoring cameras can be used for monitoring. The main monitoring targets are pedestrians, pets, luggage, etc. Considering various factors such as multiple illuminations and ground environments, it is necessary to improve the accuracy of monitoring object detection. Therefore, the embodiments of the present application provide a target detection method, which fuses images of different time periods and different perspectives to improve the accuracy of target detection.

[0046] According to the embodiments of the present application, a target detection method embodiment is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] In the present embodiment, a target detection method is provided, Figure 1 is a flowchart of the target detection method according to the embodiments of the present application, as Figure 1 shown, the flow includes the following steps:

[0048] Step S11, acquiring at least one perspective image of a target scene, the perspective including a target perspective.

[0049] The target scene is a certain fixed scene, which can be an indoor scene or an outdoor scene. There is at least one monitoring camera for the target scene to take pictures. If there are multiple monitoring cameras, the positions of the monitoring cameras are different, so the perspectives of the monitoring cameras are different, and the obtained images are different. In the present method, the target perspective is used as the reference coordinate system. The image can be one frame or multiple frames, and the time is not limited, which can include the current time and the past time.

[0050] Step S12, performing multiple fusion processing on the images of the same perspective, extracting features based on different scales for the results of each fusion processing, and taking the extracted features as input data for the next fusion processing to obtain multiple fusion processing results.

[0051] In order to improve the accuracy of the detection result, different scales of features can be extracted in each fusion, feature extraction is performed on each obtained fusion result, and the features extracted in the last fusion are combined in the next fusion. Each information passes through several layers of neural networks, which are not limited to CNN or Transformer, and is divided into several blocks from bottom to top, and each block represents output features with different levels of detailed information and global information. Each information stream has the same number of blocks, and the output feature maps of the same level blocks are consistent in the two-dimensional resolution of the plane. The data of the same level blocks are input into the fusion module for information fusion, and the output of the fusion module represents the comprehensive result of different information paths at this scale.

[0052] The images of each perspective are respectively fused, and different scales of fusion processing results are obtained for each perspective. For example, the initial image is 512x512, and different sizes of resolutions can be extracted each time, such as 512x512, 128x128, 16x16, and 4x4.

[0053] In step S13, target detection is performed on the fusion processing results of each perspective, and non-maximum suppression processing is performed on the target detection results to obtain the detection results of each perspective.

[0054] Taking one perspective as an example, target detection is performed based on the fusion processing results of each scale, and the target can be set as a pedestrian, a pet, luggage, etc. according to requirements. Non-maximum suppression processing is performed on the results of each target detection, and the detection result with the highest score is left, which includes the detection box and score of the target in the image, such as (x, y, w, h, s), wherein x and y are the coordinates of the top-left corner of the detection box, w and h are the length and width of the coordinate box, and s represents the score of the target in the box.

[0055] The above processing is performed on the image of each perspective to obtain the detection result of each perspective.

[0056] In step S14, the overlapping regions of the detection results of other perspectives and the target perspective are calculated to determine the overlapping detection results.

[0057] Other perspectives refer to perspectives other than the target perspective. For a target scene, the same target can appear in images obtained from different perspectives, and there can be an overlapping region between the images corresponding to each perspective. The overlapping regions between the detection results of other perspectives and the target perspective are calculated, and the way of calculating the overlapping regions is not limited. The overlapping detection results are obtained. The overlapping detection results are the overlapping parts of the detection results of the target perspective and the detection results of other perspectives, and the coordinate system of the overlapping detection results is consistent with the coordinate system of the detection results of the target perspective.

[0058] Step S15, the detection result of the target view is smoothed with the overlapping detection result to determine the target detection result of the target view.

[0059] The detection result of the target view is smoothed with the overlapping detection result, and the smoothing method includes but is not limited to the method of taking the mean value of the detection frame coordinate points, so as to adjust the detection result of the target view, and finally obtain the target detection result of a certain frame under the target view.

[0060] The target detection method provided in the embodiment acquires images of at least one view, performs multiple fusion processing on the images of the same view, extracts features of the result obtained by each fusion processing as input data of the next fusion processing, respectively performs target detection and non-maximum suppression processing on the results of the multiple fusion processing, obtains the detection result of each view, calculates the overlapping area of the detection result of the target view and the detection result of other views, determines the target detection result of the target view based on the overlapping detection result and the detection result of the target view. The method is not limited to using images of one view, and the images are not limited to images at a certain moment, and the accuracy of target detection is improved.

[0061] In some embodiments, the step S11 includes: Figure 1

[0062] The current moment image, the background image, at least one image in a preset time period before the current moment and interval a preset first period, and at least one image in a preset time period before the current moment and interval a preset second period corresponding to each view of the target scene are acquired respectively, the preset second period is less than the preset first period, and the background image does not contain the target.

[0063] In the embodiment, the current moment is a video frame at time t; the at least one image in a preset time period before the current moment and interval a preset first period is a long-term memory information, the preset first period is γ, and a total of m frames of images t-γ, t-2γ,..., t-mγ are adopted, and each frame interval time is γ; the at least one image in a preset time period before the current moment and interval a preset second period is a short-term memory information, the preset second period is τ, and a total of n frames of images t-τ, t-2τ,..., t-nτ are adopted, and each frame interval time is τ; the background image is an image when the target does not exist in the target scene, and the target can be a pedestrian, a pet, luggage, etc.

[0064] ​Specifically, the preset first period can be set as 1 hour, the preset time period is set as 24 hours, that is, one frame of image is collected every 1 hour before the current time t, and one or more frames of image can be stored according to requirements, and the number of stored image frames is not limited; the preset second period can be set as 0.1 second, and one frame or more frames of image is collected every 0.1 second before the current time t. According to the setting, a fixed number of images are stored, and the cached images are constantly updated as the current time changes.

[0065] The method provided in the embodiment comprehensively analyzes past long-term memory information, short-term memory information and background information, the preset first period adopted by the long-term memory information is greater than the preset second period adopted by the short-term memory information, so that the background of the target scene can be considered to change slowly and at low frequency, and the accuracy of subsequent detection results is improved.

[0066] In the case of limited computing resources, the current time image, the background image, at least one frame of image collected every preset first period within a preset time period before the current time, and at least one frame of image collected every preset second period within the preset time period before the current time are all optional image branches, which can be adjusted according to the algorithm efficiency requirements.

[0067] In some embodiments, when the multiple fusion processes include three fusion processes, the step S12 includes the following steps: Figure 1

[0068] Step S21, extracting first target features, second target features and third target features of the target scene under the same perspective based on different resolutions.

[0069] In the embodiment, taking three fusion processes as an example, the actual number of fusion processes is not limited. For the same perspective image, features under different resolutions are collected to obtain the first target features, the second target features and the third target features, wherein the image can include the current time image, the background image and the image before the current time.

[0070] Step S22, fusing the first target features to obtain a first fusion processing result.

[0071] For the same perspective, the first target features collected from different images are fused to obtain the first fusion processing result.

[0072] Step S23, performing feature extraction on the first fusion processing result based on the second target features to obtain first extracted features, and fusing the first extracted features with the second target features to obtain a second fusion processing result.

[0073] ​The first fusion processing result is subjected to feature extraction, and the extracted first feature can be consistent with the resolution of the second target feature. When the second fusion is performed, the first feature and the second target feature are fused to obtain a second fusion processing result.

[0074] In step S24, the second fusion processing result is subjected to feature extraction based on the third target feature to obtain a second extracted feature. The second extracted feature is fused with the third target feature to obtain a third fusion processing result.

[0075] The second fusion processing result is subjected to feature extraction, and the extracted second feature can be consistent with the resolution of the third target feature. When the third fusion is performed, the second extracted feature and the third target feature are fused to obtain a third fusion processing result.

[0076] In the embodiment, each view angle respectively obtains a plurality of fusion processing results after feature fusion.

[0077] In some embodiments, the step S13 in the above Figure 1 comprises the following steps:

[0078] In step S31, the first fusion processing result, the second fusion processing result and the third fusion processing result corresponding to the same view angle are respectively input into a preset detection head to obtain detection results of different scales.

[0079] The fusion processing results are respectively input into the preset detection head to obtain detection results corresponding to the fusion processing results. The detection head is a neural network method for outputting features as (x, y, w, h, s), wherein (x, y) is the coordinate of the target detection frame, w represents the width of the target detection frame, h represents the height of the target detection frame, and s represents the detection score of the target.

[0080] In step S32, non-maximum suppression processing is performed on the detection results of different scales to determine the detection results of each view angle.

[0081] After the non-maximum suppression processing is performed on the detection results output by the detection head, the detection result with the highest score is obtained, and each view angle has its corresponding detection result.

[0082] In some embodiments, the step S14 in the above Figure 1 comprises the following steps:

[0083] In step S41, the detection results of other view angles are converted to the coordinate system of the detection results of the target view angle.

[0084] The detection results of each view angle contain the detection frame coordinates of the detection target, and the coordinate systems of each view angle are different. Therefore, the detection results of other view angles need to be converted to the coordinate system of the detection results of the target view angle.

[0085] Step S42, calculating an overlapping area between the other view and the target view based on the converted detection result, to determine an overlapping detection result.

[0086] According to the converted detection result of the other view and the detection result of the target view, the overlapping area between the other view and the target view is calculated to obtain the overlapping detection result.

[0087] Specifically, the above step S41 includes:

[0088] Step S411, based on the image of the other view and the image of the target view at the same time, calculating the position of the camera corresponding to the image of the other view relative to the target camera corresponding to the image of the target view.

[0089] Taking two common cameras as an example, camera 1 is the target camera corresponding to the target view, and camera 2 is the camera corresponding to the other view. Camera 1 and camera 2 take the same scene from different angles to obtain images taken by the two cameras at any time, which are the image of the other view and the image of the target view, respectively. It is artificially judged whether there is an overlapping area between the image of the other view and the image of the target view. If there is an overlapping area, the images are input into a multi-view image stitching and registration module to match the feature points from the two images of the two views by labeling or algorithm prediction. According to the homography matrix, the detected target position in the image of the other view, i.e. the coordinates of camera 2 relative to camera 1, is calculated.

[0090] Step S412, based on the position of the camera corresponding to the image of the other view relative to the target camera corresponding to the image of the target view and the detection result of the other view, converting the detection result of the other view to the coordinate system of the detection result of the target view.

[0091] Based on the position of camera 2 relative to camera 1, the detection result of the other view corresponding to camera 2 is globally transformed to the coordinate system of the detection result of the target view, so as to convert the detection result of the other view to the coordinate system of the detection result of the target view.

[0092] In some embodiments, the above step S15 includes the following steps: Figure 1

[0093] Step S51, establishing a space-time channel for each target in the detection result of the target view.

[0094] Step S52, smoothing the space-time channel based on the overlapping detection result, removing false positive results and false negative results in the detection result, to determine the target detection result of the target view.

[0095] ​The detection result of the image at the current time t is smoothed according to the previously retained several frames of detection results, the smoothing method includes but is not limited to the method of taking the mean value of the detection frame coordinate points, so as to realize the adjustment of the detection result of the target view angle, and output the target detection result of the target view angle at the current time t.

[0096] Taking five frames of detection frames as an example, the coordinates are 0th point (1, 2), 1st point (1, 2), 2nd point (1.3, 1.8), 3rd point (1, 2), and 4th point (1, 2). Compared with the other four points, the 2nd point has a significant jump, which can be repaired to (1, 2), (1, 2), (1, 2), (1, 2), and (1, 2) after smoothing. Thus, some inaccurate predictions can be repaired with high probability.

[0097] A specific application example of the embodiment of the present application is provided below, taking a pedestrian as an example, the algorithm flow of the embodiment is as shown in Figure 2 The fixed view angle camera image input module is used for collecting images, and adjusting the resolution, brightness, and color of the images according to the subsequent algorithm needs. The long and short time information acquisition and storage module is responsible for storing the scene and field information at regular intervals. The stored information includes: background image information, current time image information, long time memory information, and short time memory information. The background image information is the image of the target scene at no one time; the current time image information is the image of the target scene at the current time; the long time memory information is the image obtained by collecting the target scene at a fixed interval, for example, one frame of image is stored every hour, which is used to calculate the average background information of the target scene in the past 24 hours; the short time memory information is the image of the N frames of time before the current time. The images obtained by the long time memory information are separated by a preset first period, and the images obtained by the short time memory information are separated by a preset second period. The preset second period is less than the preset first period.

[0098] The video space-time comparison detection module includes a preprocessing module, a human body detection module, and a post-processing module. The preprocessing module extracts the current time image, the background image, the short time memory information before the current time, and the long time memory information from the cached video for target detection. The human body detection module is composed of a multi-branch connected network, wherein the current time image branch is necessarily selected, and the background image branch, the long time memory information branch, and the short time memory information branch can be selectively configured according to needs. For example, Figure 2As shown, each branch passes through a feature extraction module, and each branch passes through a feature comparison module to compare the output of the feature extraction module of the corresponding layer. The output of each feature comparison module is merged and input into a feature fusion module. During fusion, different levels of feature maps and the fusion results of the last time are spliced, and methods such as 2D / 3D CNN, LSTM or Transformer can be used. The fusion module inputs the results into the detection head by comprehensively comparing the results of multiple branches and multiple layers. The video space-time comparison detection module can also output the foreground and background separation results of the target scene.

[0099] For the target scene, other perspectives also pass through the space-time comparison detection module, and the detection results of other perspectives are output by the independent detection network corresponding to each perspective in the space-time comparison detection module. Figure 2 The position of the camera of other perspectives relative to the camera of the target perspective is calculated by the multi-perspective splicing and registration module, and the results are output. Based on the results output by the multi-perspective splicing and registration module and the detection results of other perspectives, global perspective transformation is performed, that is, the detection results of other perspectives are converted to the coordinate system of the target perspective, and the detection results of other perspectives after conversion are calculated. According to the detection results of other perspectives after conversion and the detection results of the target perspective, the overlapping detection results are calculated by the multi-perspective overlapping area detection verification module. The overlapping detection results and the detection results output by the human body detection module of the target perspective are input into the post-processing module together. The post-processing module establishes a space-time channel for each pedestrian in the detection results, and performs smoothing processing on the channel, so as to improve the accuracy of each detection frame and remove false positive and false negative results in the detection results.

[0100] The target detection method provided in the embodiment detects pedestrians for fixed cameras, records the input of the monitoring camera of each fixed camera in different time periods, and obtains the detection results of pedestrians at the current moment by combining the images obtained by the single camera in different time periods. The overlapping areas of the multi-perspective monitoring device are compared and verified to improve the results of pedestrian detection.

[0101] The method can maximize the use of current moment information of fixed cameras, background information of unmanned environments and various time sequence information, improve the accuracy of target detection, and is not limited to using a certain time interval information or background information. The system deployment and application scalability can be improved by selecting according to the configuration and requirements.

[0102] In the embodiment, a target detection device is also provided, which is used to implement the above-mentioned embodiments and implementation manners, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0103] The embodiment provides a target detection device, which comprises the following components: Figure 3 An image acquisition module, configured to acquire images of at least one view of a target scene, wherein the view comprises a target view; A fusion processing module, configured to perform multiple fusion processes on the images of the same view, to extract features from the results of each fusion process based on different scales, and to take the extracted features as input data of the next fusion process, so as to obtain multiple fusion processing results;

[0104] A suppression processing module, configured to perform target detection on the fusion processing results of each view respectively, and to perform non-maximum suppression processing on the results of the target detection, so as to obtain detection results of each view; A smoothing processing module, configured to perform smoothing processing on the detection results of the target view and the overlapping detection results, so as to determine target detection results of the target view.

[0105] In some embodiments, the image acquisition module comprises an image acquisition unit, configured to acquire, respectively, a current time image corresponding to each view of the target scene, a background image, at least one image of a preset first period within a preset time period before the current time, and at least one image of a preset second period within the preset time period before the current time, wherein the preset second period is less than the preset first period, and the background image does not contain a target.

[0106] In some embodiments, when the multiple fusion processes comprise three fusion processes, the fusion processing module comprises: A feature extraction unit, configured to extract first target features, second target features and third target features of the target scene under the same view based on different resolutions;

[0107] A first fusion unit, configured to fuse the first target features, so as to obtain a first fusion processing result; A second fusion unit, configured to extract features from the first fusion processing result based on the second target features, so as to obtain first extracted features, and to fuse the first extracted features and the second target features, so as to obtain a second fusion processing result;

[0108] A third fusion unit, configured to extract features from the second fusion processing result based on the third target features, so as to obtain second extracted features, and to fuse the second extracted features and the third target features, so as to obtain a third fusion processing result.

[0109] In some embodiments, when the multiple fusion processes comprise three fusion processes, the fusion processing module comprises: A feature extraction unit, configured to extract first target features, second target features and third target features of the target scene under the same view based on different resolutions;

[0110] A first fusion unit, configured to fuse the first target features, so as to obtain a first fusion processing result; A second fusion unit, configured to extract features from the first fusion processing result based on the second target features, so as to obtain first extracted features, and to fuse the first extracted features and the second target features, so as to obtain a second fusion processing result;

[0111] A third fusion unit, configured to extract features from the second fusion processing result based on the third target features, so as to obtain second extracted features, and to fuse the second extracted features and the third target features, so as to obtain a third fusion processing result.

[0112] In some embodiments, when the multiple fusion processes comprise three fusion processes, the fusion processing module comprises: A feature extraction unit, configured to extract first target features, second target features and third target features of the target scene under the same view based on different resolutions;

[0113] A first fusion unit, configured to fuse the first target features, so as to obtain a first fusion processing result; A second fusion unit, configured to extract features from the first fusion processing result based on the second target features, so as to obtain first extracted features, and to fuse the first extracted features and the second target features, so as to obtain a second fusion processing result;

[0114] A third fusion unit, configured to extract features from the second fusion processing result based on the third target features, so as to obtain second extracted features, and to fuse the second extracted features and the third target features, so as to obtain a third fusion processing result.A third fusion unit is configured to perform feature extraction on the second fusion processing result based on a third target feature to obtain a second extracted feature, fuse the second extracted feature with the third target feature, and obtain a third fusion processing result.

[0115] In some embodiments, the suppression processing module comprises:

[0116] A result input unit is configured to input the first fusion processing result, the second fusion processing result, and the third fusion processing result corresponding to the same view angle into a preset detection head respectively to obtain detection results of different scales.

[0117] A suppression processing unit is configured to perform non-maximum suppression processing on the detection results of different scales to determine the detection result of each view angle.

[0118] In some embodiments, the overlap area calculation module comprises:

[0119] A coordinate conversion unit is configured to convert the detection result of the other view angle to the coordinate system of the detection result of the target view angle.

[0120] An overlap calculation unit is configured to calculate the overlap area between the other view angle and the target view angle based on the converted detection result to determine the overlap detection result.

[0121] In some embodiments, the coordinate conversion unit comprises:

[0122] A relative position calculation sub-unit is configured to calculate the position of the camera corresponding to the image of the other view angle relative to the target camera corresponding to the image of the target view angle based on the images of the other view angle and the target view angle at the same time.

[0123] A coordinate conversion sub-unit is configured to convert the detection result of the other view angle to the coordinate system of the detection result of the target view angle based on the position of the camera corresponding to the image of the other view angle relative to the target camera corresponding to the image of the target view angle and the detection result of the other view angle.

[0124] In some embodiments, the smoothing processing module comprises:

[0125] A channel establishment unit is configured to establish a space-time channel of each target in the detection result of the target view angle.

[0126] A result optimization unit is configured to perform smoothing processing on the space-time channel based on the overlap detection result to remove false positive results and false negative results in the detection result to determine the target detection result of the target view angle.

[0127] In this embodiment, the target detection device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0128] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0129] This invention also provides a computer device having the above-described features. Figure 3 The target detection device shown.

[0130] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 4 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0131] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0132] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0133] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system and applications required by at least one function. The data storage area can store data created by the computer device according to the presentation of a small program landing page, and the like. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some optional embodiments, the memory 20 can optionally include a memory disposed remotely relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0134] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state disk. The memory 20 can also include a combination of the above-mentioned kinds of memories.

[0135] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0136] The embodiments of the present application also provide a computer readable storage medium. The above-mentioned method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network downloading, so that the method described herein can be processed by such software on a storage medium using a general-purpose computer, a special-purpose processor, or programmable or special-purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state disk, and the like. Further, the storage medium can also include a combination of the above-mentioned kinds of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0137] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A target detection method characterized by, The method comprises: acquiring images of at least one view angle of a target scene, comprising: acquiring a current time image, a background image, at least one image at intervals of a preset first period within a preset time period before the current time, and at least one image at intervals of a preset second period within the preset time period before the current time corresponding to each view angle of the target scene respectively, the preset second period being less than the preset first period, the background image not containing a target, the view angle including a target view angle; performing multiple fusion processing on images of the same view angle, performing feature extraction on the result of each fusion processing based on different scales, and taking the extracted features as input data of the next fusion processing to obtain multiple fusion processing results, when the multiple fusion processing includes three fusion processing, the performing multiple fusion processing on images of the same view angle, performing feature extraction on the result of each fusion processing based on different scales, and taking the extracted features as input data of the next fusion processing to obtain multiple fusion processing results, comprising: extracting first target features, second target features and third target features of the target scene under the same view angle based on different resolutions; performing fusion on the first target features to obtain a first fusion processing result; performing feature extraction on the first fusion processing result based on the second target features to obtain first extraction features, and performing fusion on the first extraction features and the second target features to obtain a second fusion processing result; performing feature extraction on the second fusion processing result based on the third target features to obtain second extraction features, and performing fusion on the second extraction features and the third target features to obtain a third fusion processing result; performing target detection on the fusion processing results of each view angle respectively, and performing non-maximum suppression processing on the results of target detection to obtain detection results of each view angle, comprising: inputting the first fusion processing result, the second fusion processing result and the third fusion processing result corresponding to the same view angle into a preset detection head respectively to obtain detection results of different scales; performing non-maximum suppression processing on the detection results of different scales to determine the detection results of each view angle; calculating overlapping areas of the detection results of other view angles and the detection results of the target view angle to determine overlapping detection results; performing smoothing processing on the detection results of the target view angle and the overlapping detection results to determine target detection results of the target view angle.

2. The method of claim 1, wherein, The calculation of the overlapping areas of the detection results of other view angles and the detection results of the target view angle to determine the overlapping detection results comprises: converting the detection results of other view angles to the coordinate system of the detection results of the target view angle; calculating the overlapping areas of other view angles and the target view angle based on the converted detection results to determine the overlapping detection results.

3. The method of claim 2, wherein, The conversion of the detection results of other view angles to the coordinate system of the detection results of the target view angle comprises: based on the images of other view angles and the images of the target view angle at the same time, calculating the positions of the cameras corresponding to the images of other view angles relative to the target camera corresponding to the images of the target view angle; The detection result of the other view is converted to a coordinate system of the target view detection result based on a position of a camera corresponding to the other view relative to a target camera corresponding to the target view and the detection result of the other view.

4. The method of claim 1, wherein, The target view detection result and the overlap detection result are smoothed to determine a target view target detection result. A time-space channel of each target in the target view detection result is established. The time-space channel is smoothed based on the overlap detection result to remove false positive results and false negative results in the detection result to determine the target view target detection result.

5. A target detection apparatus characterized by comprising: The device for performing the target detection method of any one of claims 1 to 4 comprises: An image acquisition module acquires images of at least one view of a target scene, the view including a target view; the image acquisition module includes an image acquisition unit that acquires a current time image, a background image, at least one image at intervals of a preset first period within a preset time period before the current time, and at least one image at intervals of a preset second period within the preset time period before the current time corresponding to each view of the target scene, the preset second period being less than the preset first period, and the background image not containing a target; A fusion processing module performs multiple fusion processes on images of the same view, extracts features based on different scales for the result of each fusion process, and uses the extracted features as input data for the next fusion process to obtain multiple fusion processing results; when the multiple fusion processes include three fusion processes, the fusion processing module includes: a feature extraction unit that extracts first target features, second target features, and third target features of the target scene under the same view based on different resolutions; a first fusion unit that fuses the first target features to obtain a first fusion processing result; a second fusion unit that extracts features based on the second target features for the first fusion processing result to obtain first extracted features, and fuses the first extracted features with the second target features to obtain a second fusion processing result; and a third fusion unit that extracts features based on the third target features for the second fusion processing result to obtain second extracted features, and fuses the second extracted features with the third target features to obtain a third fusion processing result; A suppression processing module performs target detection on the fusion processing results of each view respectively, and performs non-maximum suppression processing on the results of the target detection to obtain detection results of each view; the suppression processing module includes: A result input unit inputs the first fusion processing result, the second fusion processing result, and the third fusion processing result corresponding to the same view into a preset detection head respectively to obtain detection results of different scales; a suppression processing unit performs non-maximum suppression processing on the detection results of different scales to determine the detection results of each view; An overlap region calculation module calculates an overlap region of the detection results of other views and the detection results of the target view to determine overlap detection results. The smoothing processing module is configured to perform smoothing processing on the detection result of the target view and the overlap detection result, to determine a target detection result of the target view.

6. A computer device, comprising: The method comprises the following steps: The memory and the processor are connected in communication with each other, and the memory stores computer instructions. The processor executes the computer instructions to perform the target detection method in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to perform the target detection method in any one of claims 1 to 4.

Images