Target detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a target detection method and device, electronic equipment and a storage medium. The method comprises the following steps: determining target motion velocity vector information of a target object in a first shooting picture; target time consumption information from the start of a first shot picture to the formation of a second shot picture is determined, the first shot picture is formed by splicing a plurality of imaging pictures of a multi-view camera, and the second shot picture is formed by splicing a plurality of imaging pictures of the multi-view camera after the first shot picture; determining a predicted motion position of the target object in a to-be-formed second shooting picture according to the target motion velocity vector information and the target time consumption information; and determining a target position frame for tracking the target object in the formed second shot picture based on the predicted motion position when the second shot picture is formed. According to the scheme, the problems of jumping and lagging of the target position frame of target detection in the shooting scene of the multi-view camera are solved, and the detection accuracy of the target object in the shot picture is improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of the video security industry, camera models have also shown characteristics such as diversification, refinement, and customization. In order to meet the market demand for a single device to cover a wide area, multi-camera products represented by binocular and quad-camera cameras came into being. Multi-cameras have the characteristics of using fewer devices in the same monitoring scene, which not only meets the requirements of beautiful use environment, easy operation, but also saves costs. However, the multi-camera products have the characteristics of multiple imaging sensor image fusion, which causes the target frame to frequently jump left and right and lag during the target detection process, affecting the detection effect. Summary of the invention

[0003] The present invention provides a target detection method, device, electronic device and storage medium to solve the problem of target position frame jump and lag in target detection in a multi-camera shooting scene.

[0004] According to one aspect of the present invention, a target detection method is provided, the method comprising:

[0005] Determine target motion velocity vector information of a target object in a first captured image;

[0006] Determine target time information from the first shot picture to the formation of a second shot picture, the first shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera, and the second shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera after the first shot picture;

[0007] Determining a predicted motion position of the target object in a second shooting picture to be formed according to the target motion speed vector information and the target time-consuming information;

[0008] When forming a second photographing picture, a target position frame for tracking the target object is determined in the formed second photographing picture based on the predicted motion position.

[0009] According to another aspect of the present invention, there is provided a target detection device, the device comprising:

[0010] A first determining module, used to determine target motion speed vector information of a target object in a first shooting picture;

[0011] A second determination module is used to determine target time-consuming information from the first shooting picture to the formation of a second shooting picture, the first shooting picture is formed by stitching together a plurality of imaging pictures of a multi-camera, and the second shooting picture is formed by stitching together a plurality of imaging pictures of a multi-camera after the first shooting picture;

[0012] A third determination module is used to determine the predicted motion position of the target object in the second shooting picture to be formed according to the target motion speed vector information and the target time-consuming information;

[0013] The fourth determining module determines a target position frame for tracking the target object in the formed second shooting picture based on the predicted motion position when forming the second shooting picture.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the target detection method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention determines the target motion speed vector information of the target object in the first shooting picture and determines the target time information from the first shooting picture to the formation of the second shooting picture. Then, during the period from the first shooting picture to the formation of the second shooting picture, the motion position of the target object detected in the first shooting picture can be corrected according to the target motion speed vector information and the target time information, so as to obtain the predicted motion position that the target object may reach when the second shooting picture is formed as accurately as possible, and then when the second shooting picture is formed, the target position frame of the tracked target object can be marked in the second shooting picture, which solves the problem of target position frame jumping and lagging in target detection in multi-camera shooting scenes, and improves the detection accuracy of the target object in the shooting picture.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a target detection method provided according to an embodiment of the present invention;

[0023] Figure 2 It is a schematic diagram of position frame marking of a shooting picture generated by splicing and fusing multiple imaging pictures of a multi-camera in the prior art to which an embodiment of the present invention is applicable;

[0024] Figure 3a It is a schematic diagram of marking the moving position of a target object in a shooting picture of a multi-camera in the prior art to which an embodiment of the present invention is applicable;

[0025] Figure 3b It is a schematic diagram of another method of marking the moving position of a target object in a shooting picture of a multi-camera in the prior art to which an embodiment of the present invention is applicable;

[0026] Figure 3c It is a schematic diagram of marking the moving position of a target object in a shooting picture of a multi-camera in another prior art to which an embodiment of the present invention is applicable;

[0027] Figure 3d It is a schematic diagram of marking the moving position of a target object in a shooting picture of a multi-camera in another prior art to which an embodiment of the present invention is applicable;

[0028] Figure 4a It is a schematic diagram of the principle of marking a position frame of a target object in a shooting picture of a multi-camera according to an embodiment of the present invention;

[0029] Figure 4b It is another schematic diagram of the principle of marking a position frame of a target object in a shooting picture of a multi-camera according to an embodiment of the present invention;

[0030] Figure 5 is a flow chart of another target detection method provided according to an embodiment of the present invention;

[0031] Figure 6 is a schematic structural diagram of a target detection device provided according to an embodiment of the present invention;

[0032] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Figure 1 A flow chart of a target detection method is provided for an embodiment of the present invention. This embodiment can be applicable to the situation of tracking and detecting target objects in the shooting pictures of multi-eye cameras. The method can be executed by a target detection device, which can be implemented in the form of hardware and / or software. The target detection device can be configured in any electronic device with network communication function, which electronic device can be a multi-eye camera, etc.

[0036] like Figure 1 As shown, the target detection method in this embodiment may include the following process:

[0037] S110, determining target motion velocity vector information of a target object in a first shooting picture, where the first shooting picture is formed by splicing a plurality of imaging pictures of a multi-camera.

[0038] The multi-eye camera is equipped with multiple lenses, which can be distributed in sequence along the circumferential direction of the camera. Each lens can be independently adjusted in three axes to achieve wide-angle or specific angle image capture. The multi-eye camera can be a multi-eye network camera, etc.

[0039] For a multi-eye camera, the corresponding imaging pictures on each imaging sensor in the multi-eye camera can be received through the video input device components vif dev0 and vif dev2, and the corresponding imaging pictures on each imaging sensor can be spliced ​​and fused into a complete picture through the image processing engine, which can be the shooting picture of the multi-eye camera. Among them, the image processing engine supports image quality adjustment of an input image, including but not limited to noise reduction, sharpening, brightness adjustment, etc., and then scaled to a certain resolution and output through each output interface.

[0040] The target object may refer to various objects in motion, including a moving car, a walking pedestrian, and other people or objects in motion. The first shot picture may be formed by splicing and fusing corresponding imaging pictures on multiple imaging sensors of a multi-camera, and the first shot picture includes the target object. The target motion speed vector information may be the motion speed and motion direction of the target object in the first shot picture.

[0041] S120, determining target time information from the first shooting picture to the formation of a second shooting picture, where the second shooting picture is formed by stitching together a plurality of imaging pictures of the multi-camera after the first shooting picture.

[0042] The multi-camera will detect in real time to obtain continuous shooting pictures. The second shooting picture is a new shooting picture that will be formed by stitching and fusing the corresponding imaging pictures on multiple imaging sensors of the multi-camera after the multi-camera forms the first shooting picture. From the first shooting picture formed by the multi-camera to the formation of the second shooting picture, it involves the operation of detecting the moving position of the target object in the shooting picture of the multi-camera and stitching and fusing the corresponding imaging pictures on multiple imaging sensors. These operations take a certain amount of time, and the target time from the first shooting picture to the formation of the second shooting picture is obtained.

[0043] S130 , determining a predicted motion position of the target object in a second captured image to be formed according to the target motion speed vector information and the target time consumption information.

[0044] See also Figure 2,For the problem of target position frame jumping and lagging, in the traditional solution, after the corresponding imaging images on the two imaging sensors of the multi-eye camera are spliced ​​and fused to form a shooting picture, it is usually necessary to perform motion position detection in sequence according to the left and right areas to obtain the target position frame for the target object in the shooting picture. However, the detection mode of left first then right and right first then left will cause the target position frame to jump left and right due to the difference in detection order, which is not conducive to the observation of the real-time effect and reduces the detection rate. In addition, the time for left and right detection is long. After the detection is completed, there is a difference between the marked moving object detection frame and the current position of the object. The faster the target object moves, the greater the difference. It may not be possible to mark the target position frame on the shooting picture in time, causing the target position frame to lag. See Figure 3a-3d The figure shows the target position frame marking schematic diagram of multiple consecutive frames of shooting pictures in the traditional solution.

[0045] In addition, the method of stitching and fusing the corresponding imaging images on the two imaging sensors of the multi-eye camera to form a shooting picture, and uniformly detecting the motion position of the target object in the shooting picture, because the detection is performed using the complete shooting picture after fusion, has high CPU performance requirements and may cause high computing costs. Moreover, as the number of "eyes" of the camera increases and the resolution increases, the unified detection rate after stitching and fusing multiple frames of imaging pictures will decrease, resulting in a more serious target position frame lag problem than before, and even causing excessive CPU load in the camera.

[0046] Based on the above situation, for the motion position detection of the target object in the first shooting picture of the multi-eye camera, due to the long detection time, it may be impossible to obtain the position detection in time and thus fail to mark the target position frame in the first shooting picture. At the same time, combined with the actual application scenarios of the multi-eye camera, it is not difficult to find that most moving target objects will not have sudden changes in speed and direction in a short period of time. Therefore, after determining the target motion speed vector information and target time information of the target object from the first shooting picture, the target motion speed vector information and target time information can be used to predict the position of the target object in the second shooting picture before forming the second shooting picture, so that the motion position detection of the target object in the second shooting picture can be carried out in time.

[0047] As an optional but non-limiting implementation method, the target timing information is determined based on the total detection time for motion position detection of the target object in the shooting picture of the multi-eye camera and the fusion time required for splicing and fusing multiple imaging pictures of the multi-eye camera to form a second shooting picture after the first shooting picture. The total detection time is the sum of the detection time for motion position detection of each sub-picture area sequentially divided in the first shooting picture.

[0048] See also Figure 4a, the time from the start of the first shooting picture to the formation of the second shooting picture can be determined as the target time, so that the motion position predicted by using the target time and the target motion velocity vector is exactly the same as the motion position obtained by real-time detection of the motion position of the target object in the second shooting picture.

[0049] See also Figure 4a The target time consumption can be calculated by combining the time consumed by the operation of detecting the motion position of the target object in the shooting picture of the multi-camera and the operation of splicing and fusing the corresponding imaging pictures on multiple imaging sensors. The total detection time consumption of the motion position detection of the target object can be the sum of the detection time consumption of the motion position detection of each sub-picture area sequentially divided in the first shooting picture from left to right.

[0050] Optionally, the time duration indicated by the target time duration information from the first shot to the formation of the second shot is greater than or equal to the sum of the total detection time for detecting the motion position of the target object in the shot of the multi-camera and the fusion time required for the second shot to be formed by splicing and fusing multiple imaging pictures of the multi-camera after the first shot. It is understandable that the target time duration should not be too long to prevent the shooting environment from being judged inaccurately due to a long time period.

[0051] As an optional but non-limiting implementation, determining the predicted motion position of the target object in the second captured image to be formed according to the target motion velocity vector information and the target time-consuming information may include the following steps A1-A2:

[0052] Step A1: determine the target motion position of the target object in the first shooting picture.

[0053] Step A2: determine the first predicted movement distance of the target object in the shooting picture of the multi-camera during the reference movement period according to the target movement speed vector information and the target time information, where the reference movement period is the time period from the first shooting picture to the formation of the second shooting picture.

[0054] Step A3: determining the predicted motion position of the target object in the second captured image to be formed according to the target motion position and the first predicted motion distance.

[0055] The target motion position may be the position of the target object in the first shot picture, for example, the target motion position of the target object may be marked in the first shot picture by a target position frame. The motion position of the target object in the first shot picture may be detected by using one or more of the following methods including but not limited to: a method based on clustering theory, a method based on fuzzy theory, a method based on statistical theory, a background modeling method, a method based on a neural network, an optical flow method, a frame difference method, etc.

[0056] See also Figure 4a and Figure 4b , from the first shooting picture to the time period when the second shooting picture is formed, the moving distance of the target object in the shooting picture of the multi-eye camera can be determined by calculating the target motion velocity vector information and the target time information. After the moving distance of the target object in the shooting picture of the multi-eye camera is obtained, it can be superimposed on the moving distance of the target object in the shooting picture of the multi-eye camera according to the target motion position of the target object in the first shooting picture, so that before the second shooting picture is formed, the moving position of the target object in the first shooting picture before the second shooting picture can be used to predict the moving position of the target object in the second shooting picture.

[0057] Optionally, the first shot picture and the second shot picture can be two adjacent frames of shot pictures formed by stitching together multiple imaging pictures of a multi-camera in succession, or two frames of shot pictures separated by a preset number of frames, and it is necessary to ensure that the target time from the first shot picture to the formation of the second shot picture is greater than or equal to the sum of the total detection time for motion position detection of the target object in the shooting picture of the multi-camera and the fusion time required for stitching and fusing multiple imaging pictures of the multi-camera after the first shot picture to form the second shot picture.

[0058] In some embodiments, the positions of the monocular cameras that make up the multi-eye camera are relatively fixed (such as an integrated multi-eye / binocular camera), and the stitching algorithm is based on cropping, correction and stitching of fixed coordinates, so the fusion time of each frame is basically the same; in other embodiments, the monocular cameras that make up the multi-eye camera are relatively independent, and the stitching algorithm is based on automatic recognition and matching of feature points, and then cropping, correction and stitching are performed, so the fusion time is affected by the difficulty of feature point matching. In the above embodiment, when the motion position of the target object in the second shooting picture is predicted and calculated, the target time can be adaptively adjusted in conjunction with the actual scene. For example, when the detailed texture in the scene is less or too cluttered (such as a water surface scene), the target time is extended by 20% compared with other scenes.

[0059] By adopting the above method, since the movement position of the target object in the first shooting picture before the second shooting picture is formed is used to predict the movement position of the target object in the second shooting picture to be formed, as the "number of eyes" of the multi-eye camera increases, the lag and jump phenomenon will not be more obvious the faster the moving object is, and the detection efficiency can be improved.

[0060] As an optional but non-limiting implementation, determining the first predicted movement distance of the target object in the shooting picture of the multi-camera during the reference movement according to the target movement speed vector information and the target time information may include the following steps B1-B2:

[0061] Step B1: Determine a second predicted movement distance of the target object in a real scene area corresponding to the shooting picture of the multi-camera during the reference movement period according to the target movement speed vector information and the target time consumption information.

[0062] Step B2: Determine the first predicted motion distance of the target object in the shooting picture of the multi-camera during the reference movement based on the second predicted motion distance and the preset motion distance mapping information. The preset motion distance mapping information is used to describe the distances of the shooting picture of the multi-camera corresponding to different distances of the real scene area mapped to the shooting picture of the multi-camera based on the pre-calibration of the focal length of the multi-camera.

[0063] See also Figure 4b , the movement speed and movement direction indicated in the target movement speed vector information are respectively the position of the target moving object in the first shooting picture and the movement direction of the target object relative to the preset reference object in the shooting picture of the multi-eye camera. The movement speed and movement direction indicated in the target movement speed vector information describe the movement speed and movement direction of the target object in the real scene area corresponding to the shooting picture of the multi-eye camera. Furthermore, the target movement speed vector information and the target time information can be used to calculate the second predicted movement distance of the target object in the real scene area corresponding to the shooting picture of the multi-eye camera during the reference movement.

[0064] Since the real scene area corresponding to the shooting picture of the multi-eye camera is not in proportional relationship with the shooting picture of the multi-eye camera, it is necessary to determine the mapping relationship between the different distances of the real scene area corresponding to the shooting picture of the multi-eye camera and the distances of the shooting picture of the multi-eye camera, and then use the mapping relationship to map the second predicted motion distance from the real scene area corresponding to the shooting picture of the multi-eye camera to the shooting picture of the multi-eye camera, so as to obtain the first predicted motion distance of the target object in the shooting picture of the multi-eye camera during the reference movement.

[0065] Exemplarily, assuming that the target motion speed vector information indicates that the motion speed of the target object in the motion direction is v, and the target time information indicates that the time from the first shooting picture to the formation of the second shooting picture is t seconds, then the target object has moved s(m)=v*t during the period from the first shooting picture to the formation of the second shooting picture, and then the preset motion distance mapping information is queried according to the focal length of the multi-eye camera, and the ratio of the distance of the shooting picture corresponding to the real scene area to the distance of the shooting picture indicated by the preset motion distance mapping information is set to k; it can be inferred that the first predicted motion distance of the target object in the shooting picture of the multi-eye camera during the reference movement is S=k*s, and then the predicted motion position of the target object on the second shooting picture to be formed can be obtained according to the first predicted motion distance.

[0066] S140 . When forming a second photographing picture, determine a target position frame for tracking the target object in the formed second photographing picture based on the predicted motion position.

[0067] When determining the target motion position of the target object in the first captured picture, the size of the target object can also be determined at the same time. After predicting the predicted motion position of the target object in the second captured picture, a target position frame that matches the size of the target object can be marked at the corresponding predicted position based on the predicted motion position and the size of the target object. In this way, when displaying the second captured picture, the target position frame can be used on the second captured picture to track the target object in the second captured picture. The target position frame can be a rectangular frame.

[0068] The technical solution of the embodiment of the present invention determines the target motion speed vector information of the target object in the first shooting picture and determines the target time information from the first shooting picture to the formation of the second shooting picture. Then, during the period from the first shooting picture to the formation of the second shooting picture, the motion position of the target object detected in the first shooting picture can be corrected according to the target motion speed vector information and the target time information, so as to obtain the predicted motion position that the target object may reach when the second shooting picture is formed as accurately as possible, and then when the second shooting picture is formed, the target position frame of the tracked target object can be marked in the second shooting picture, which solves the problem of target position frame jumping and lagging in target detection in multi-camera shooting scenes, and improves the detection accuracy of the target object in the shooting picture.

[0069] Figure 5 A flowchart of another target detection method is provided for an embodiment of the present invention. The technical solution of this embodiment further optimizes the process of determining the target motion speed vector information of the target object in the first shooting picture in the aforementioned embodiment on the basis of the aforementioned embodiment. This embodiment can be combined with various optional solutions in one or more of the aforementioned embodiments.

[0070] like Figure 5 As shown, the target detection method in this embodiment may include the following process:

[0071] S510, determining target motion state information of a target object in a first captured image, where the target motion state information is a first motion state or a second motion state, and when the target object is in the first motion state, a motion distance within a reference time period is not less than a reference motion distance, and when the target object is in the second motion state, a motion distance within a reference time period is less than a reference motion distance.

[0072] For a multi-camera, the motion state of the target object entering the shooting picture of the multi-camera is not unique. The motion state of the target object can be described as the speed of the target object in the shooting picture. For example, some target objects move very fast, and such target objects will move a large distance in a small number of frames. If the detection time of the moving position of the target object in the shooting picture is relatively long, it will not be possible to mark the target position frame in the shooting picture. However, some target objects move very slowly, and such target objects will basically not move a large distance in a small number of frames. It can be almost considered that the moving position of the target object in multiple consecutive frames is consistent.

[0073] Based on the above analysis, when using a multi-camera, the motion position of the target object in the first shooting picture is not always used in combination with the motion speed to predict the motion position of the target object in the second shooting picture to be formed. Instead, it is necessary to consider the target motion state information of the target object in the first shooting picture to judge the speed of movement of the target object in the shooting picture, so as to decide whether to start using the motion position of the target object in the first shooting picture in combination with the motion speed to predict the motion position of the target object in the second shooting picture to be formed.

[0074] As an optional but non-limiting implementation, determining the target motion state information of the target object in the first captured image may include the following steps C1-C2:

[0075] Step C1: after detecting that the target object enters the shooting screen of the multi-eye camera, determine at least two frames of the third shooting screen, where the at least two frames of the third shooting screen are formed by continuously stitching together multiple imaging screens of the multi-eye camera and include a preset number of frames of the first shooting screen.

[0076] Based on the fused shooting pictures of the multi-camera, multiple imaging pictures that continuously pass through the multi-camera including the target object are spliced ​​to form a third shooting picture. The first 5-10 frames of the shooting picture when the target object first enters the shooting picture of the multi-camera can be used as the third shooting picture and analyzed and the movement direction and speed of the target object are recorded. According to the movement direction and speed of the target object, it is judged whether the target object is in the first motion state or the second motion state.

[0077] Step C2: determining the target motion position of the target object in each frame of the third shooting picture, and determining the target motion state information of the target object in the third shooting picture according to each target motion position.

[0078] Optionally, determining the target motion state information of the target object in the third shooting picture based on each target motion position may include: if the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is greater than or equal to the reference motion distance, then determining the target motion state information as the first motion state; if the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is less than the reference motion distance, then determining the target motion state information as the second motion state, and the reference time length is the interval time length between two adjacent frames of the third shooting picture.

[0079] S520: If the target motion state information is the first motion state, determine the target motion speed vector information of the target object in the first shooting picture, where the motion speed vector information includes a motion speed and a motion direction.

[0080] Based on the above embodiment, optionally, after determining the target object motion state information in the first shooting picture, the following steps D1-D2 may be further included:

[0081] Step D1: If the target motion state information is the second motion state, determine the target motion position of the target object in the first shooting picture.

[0082] Step D2: when forming a second shooting picture, determining a target position frame for tracking the target object in the formed second shooting picture based on the target motion position.

[0083] Exemplarily, since the movement of the target object in the shooting picture of the multi-eye camera may be uncertain, the target object of each frame of the third shooting picture is collected during the movement process, the influence of environmental changes in each frame of the third shooting picture is eliminated, and the two adjacent frames of the third shooting pictures are compared, and the comparison is performed 5 times per second. When the moving distance of the target object within the reference time length for 5 consecutive times is less than the reference movement distance, the target object is determined to be in a "slow motion state". At this time, the target motion speed vector information and the target time information are not used to predict the position of the target object in the second shooting picture, and the target position frame is marked in real time. At this time, in the "slow motion state", the multi-eye fusion time consumption can be ignored. When the moving distance of the target object within the reference time length for 5 consecutive times is greater than or equal to the reference movement distance, the target object is determined to be in a "fast motion state". At this time, the target motion speed vector information and the target time information are used to predict the position of the target object in the second shooting picture.

[0084] S530, determining target time information from the first shooting picture to the formation of the second shooting picture, the first shooting picture is formed by stitching together multiple imaging pictures of the multi-eye camera, and the second shooting picture is formed by stitching together multiple imaging pictures of the multi-eye camera after the first shooting picture.

[0085] S540: Determine the predicted motion position of the target object in the second captured image to be formed according to the target motion speed vector information and the target time consumption information.

[0086] S550: When forming a second photographing picture, determine a target position frame for tracking the target object in the formed second photographing picture based on the predicted motion position.

[0087] The technical solution of the embodiment of the present invention determines the target motion speed vector information of the target object in the first shooting picture and determines the target time information from the first shooting picture to the formation of the second shooting picture. Then, during the period from the first shooting picture to the formation of the second shooting picture, the motion position of the target object detected in the first shooting picture can be corrected according to the target motion speed vector information and the target time information, so as to obtain the predicted motion position that the target object may reach when the second shooting picture is formed as accurately as possible, and then when the second shooting picture is formed, the target position frame of the tracked target object can be marked in the second shooting picture, which solves the problem of target position frame jumping and lagging in target detection in multi-camera shooting scenes, and improves the detection accuracy of the target object in the shooting picture.

[0088] Figure 6A structural block diagram of a target detection device is provided for an embodiment of the present invention. This embodiment can be applied to the situation of tracking and detecting target objects in the shooting pictures of multi-eye cameras. The target detection device can be implemented in the form of hardware and / or software. The target detection device can be configured in any electronic device with network communication function, which can be a multi-eye camera, etc.

[0089] like Figure 6 As shown, the target detection device in this embodiment may include the following process:

[0090] A first determining module 610 is used to determine target motion speed vector information of a target object in a first captured image;

[0091] A second determining module 620 is used to determine target time information from the first shot picture to the formation of a second shot picture, the first shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera, and the second shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera after the first shot picture;

[0092] A third determination module 630 is used to determine the predicted motion position of the target object in the second shooting picture to be formed according to the target motion speed vector information and the target time consumption information;

[0093] The fourth determining module 640 determines a target position frame for tracking the target object in the formed second shooting picture based on the predicted motion position when forming the second shooting picture.

[0094] Based on the above embodiment, optionally, the target time information is determined based on the total detection time for motion position detection of the target object in the shooting picture and the fusion time required for stitching and fusing multiple imaging pictures of the multi-camera to form a second shooting picture. The total detection time is the sum of the detection times for motion position detection of each sub-picture area sequentially divided in the first shooting picture.

[0095] Based on the above embodiment, optionally, determining target motion speed vector information of the target object in the first shooting picture includes:

[0096] Determine target motion state information of a target object in a first captured image, wherein the target motion state information is a first motion state or a second motion state, a motion distance of the target object within a reference time period when the target object is in the first motion state is not less than a reference motion distance, and a motion distance of the target object within a reference time period when the target object is in the second motion state is less than the reference motion distance;

[0097] If the target motion state information is the first motion state, target motion speed vector information of the target object in the first shooting picture is determined, and the motion speed vector information includes a motion speed and a motion direction.

[0098] Based on the above embodiment, optionally, determining the target motion state information of the target object in the first captured picture includes:

[0099] After detecting that the target object enters the shooting picture of the multi-eye camera, determining at least two frames of the third shooting picture, wherein the at least two frames of the third shooting picture are formed by continuously stitching together a plurality of imaging pictures of the multi-eye camera and include a preset number of frames of the first shooting picture;

[0100] The target motion position of the target object in each frame of the third shooting picture is determined, and the target motion state information of the target object in the third shooting picture is determined according to each target motion position.

[0101] On the basis of the above embodiment, optionally, determining the target motion state information of the target object in the third shooting picture according to each target motion position includes:

[0102] If the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is greater than or equal to the reference motion distance, determining that the target motion state information is the first motion state;

[0103] If the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is less than the reference motion distance, the target motion state information is determined to be the second motion state, and the reference duration is the interval duration between two adjacent frames of the third shooting picture.

[0104] Based on the above embodiment, optionally, after determining the target object motion state information in the first shooting picture, the method further includes:

[0105] If the target motion state information is the second motion state, determining the target motion position of the target object in the first shooting picture;

[0106] When forming the second photographing picture, a target position frame for tracking the target object is determined in the formed second photographing picture based on the target motion position.

[0107] On the basis of the above embodiment, optionally, determining the predicted motion position of the target object in the second captured picture to be formed according to the target motion speed vector information and the target time-consuming information includes:

[0108] Determining a target motion position of a target object in a first captured image;

[0109] Determine, according to the target motion speed vector information and the target time-consuming information, a first predicted motion distance of the target object in the shooting picture of the multi-camera during a reference motion period, wherein the reference motion period is a time period from the start of the first shooting picture to the formation of a second shooting picture;

[0110] The predicted movement position of the target object in a second captured image to be formed is determined according to the target movement position and the first predicted movement distance.

[0111] On the basis of the above embodiment, optionally, determining a first predicted movement distance of the target object in a shooting picture of a multi-camera during a reference movement according to the target movement speed vector information and the target time-consuming information includes:

[0112] Determine, according to the target motion speed vector information and the target time-consuming information, a second predicted motion distance of the target object in the real scene area corresponding to the shooting picture of the multi-camera during the reference movement period;

[0113] Based on the second predicted motion distance and the preset motion distance mapping information, the first predicted motion distance of the target object in the shooting picture of the multi-eye camera during the reference movement is determined. The preset motion distance mapping information is used to describe the distances mapped to the shooting picture of the multi-eye camera from different distances of the real scene area corresponding to the shooting picture of the multi-eye camera which is pre-calibrated based on the focal length of the multi-eye camera.

[0114] The technical solution of the embodiment of the present invention determines the target motion speed vector information of the target object in the first shooting picture and determines the target time information from the first shooting picture to the formation of the second shooting picture. Then, during the period from the first shooting picture to the formation of the second shooting picture, the motion position of the target object detected in the first shooting picture can be corrected according to the target motion speed vector information and the target time information, so as to obtain the predicted motion position that the target object may reach when the second shooting picture is formed as accurately as possible, and then when the second shooting picture is formed, the target position frame of the tracked target object can be marked in the second shooting picture, which solves the problem of target position frame jumping and lagging in target detection in multi-camera shooting scenes, and improves the detection accuracy of the target object in the shooting picture.

[0115] The target detection device provided in the embodiment of the present invention can execute the target detection method provided in any embodiment of the present invention mentioned above, and has the corresponding functions and beneficial effects of executing the target detection method. For detailed process, please refer to the relevant operations of the target detection method in the above embodiment.

[0116] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0117] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0118] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0119] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a target detection method.

[0121] In some embodiments, the target detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the target detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the target detection method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Determine target motion velocity vector information of a target object in a first captured image; Determine target time information from the first shot picture to the formation of a second shot picture, the first shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera, and the second shot picture is formed by stitching together a plurality of imaging pictures of a multi-camera after the first shot picture; Determining a predicted motion position of the target object in a second shooting picture to be formed according to the target motion speed vector information and the target time-consuming information; When forming the second photographing picture, a target position frame for tracking the target object is determined in the formed second photographing picture based on the predicted motion position.

2. The method according to claim 1, characterized in that The target time information is determined based on the total detection time for motion position detection of the target object in the shooting picture and the fusion time required for splicing and fusing multiple imaging pictures of the multi-camera to form a second shooting picture. The total detection time is the sum of the detection times for motion position detection of each sub-picture area sequentially divided in the first shooting picture.

3. The method according to claim 1, characterized in that: Determining target motion velocity vector information of a target object in a first captured image includes: Determine target motion state information of a target object in a first captured image, wherein the target motion state information is a first motion state or a second motion state, a motion distance of the target object within a reference time period when the target object is in the first motion state is not less than a reference motion distance, and a motion distance of the target object within a reference time period when the target object is in the second motion state is less than the reference motion distance; If the target motion state information is the first motion state, target motion speed vector information of the target object in the first shooting picture is determined, and the motion speed vector information includes a motion speed and a motion direction.

4. The method according to claim 3, characterized in that Determining target motion state information of a target object in a first captured image includes: After detecting that the target object enters the shooting picture of the multi-eye camera, determining at least two frames of the third shooting picture, wherein the at least two frames of the third shooting picture are formed by continuously stitching together a plurality of imaging pictures of the multi-eye camera and include a preset number of frames of the first shooting picture; The target motion position of the target object in each frame of the third shooting picture is determined, and the target motion state information of the target object in the third shooting picture is determined according to each target motion position.

5. The method according to claim 4, characterized in that Determining target motion state information of the target object in the third shooting picture according to each target motion position includes: If the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is greater than or equal to the reference motion distance, determining that the target motion state information is the first motion state; If the position distance difference between the target motion positions corresponding to two adjacent frames of the third shooting picture is less than the reference motion distance, the target motion state information is determined to be the second motion state, and the reference duration is the interval duration between two adjacent frames of the third shooting picture.

6. The method according to claim 3, characterized in that After determining the target object motion state information in the first shooting picture, the method further includes: If the target motion state information is the second motion state, determining the target motion position of the target object in the first shooting picture; When forming the second photographing picture, a target position frame for tracking the target object is determined in the formed second photographing picture based on the target motion position.

7. The method according to claim 1, characterized in that Determining the predicted motion position of the target object in a second shooting picture to be formed according to the target motion speed vector information and the target time-consuming information includes: Determining a target motion position of a target object in a first captured image; Determine, according to the target motion speed vector information and the target time-consuming information, a first predicted motion distance of the target object in the shooting picture of the multi-camera during a reference motion period, wherein the reference motion period is a time period from the start of the first shooting picture to the formation of a second shooting picture; The predicted movement position of the target object in a second captured image to be formed is determined according to the target movement position and the first predicted movement distance.

8. The method according to claim 7, characterized in that Determining a first predicted movement distance of the target object in a shooting picture of a multi-camera during a reference movement according to the target movement speed vector information and the target time-consuming information, comprising: Determine, according to the target motion speed vector information and the target time-consuming information, a second predicted motion distance of the target object in the real scene area corresponding to the shooting picture of the multi-camera during the reference movement period; Based on the second predicted motion distance and the preset motion distance mapping information, the first predicted motion distance of the target object in the shooting picture of the multi-eye camera during the reference movement is determined. The preset motion distance mapping information is used to describe the distances mapped to the shooting picture of the multi-eye camera from different distances of the real scene area corresponding to the shooting picture of the multi-eye camera which is pre-calibrated based on the focal length of the multi-eye camera.

9. A target detection device, characterized in that: The device comprises: A first determining module, used to determine target motion speed vector information of a target object in a first shooting picture; A second determination module is used to determine target time-consuming information from the first shooting picture to the formation of a second shooting picture, the first shooting picture is formed by stitching together a plurality of imaging pictures of a multi-camera, and the second shooting picture is formed by stitching together a plurality of imaging pictures of a multi-camera after the first shooting picture; A third determination module is used to determine the predicted motion position of the target object in the second shooting picture to be formed according to the target motion speed vector information and the target time-consuming information; The fourth determining module determines a target position frame for tracking the target object in the formed second shooting picture based on the predicted motion position when forming the second shooting picture.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the target detection method according to any one of claims 1 to 8 when executed.

Images