Method, device, electronic device and storage medium for determining moving objects in images

Abstract

The present application proposes a method, device, electronic device and storage medium for determining a moving object in an image, wherein the method includes: in the process of determining a target frame of a moving target object in a current frame video image, determining multiple first candidate frames of the target object in the current frame video image, and determining a moving foreground image of the current frame video image, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and filtering multiple first candidate frames to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, and while accurately determining the target frame of the moving target object in the current frame video image, the computational efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, electronic device and storage medium for determining a moving object in an image. Background Art

[0002] In recent years, with the continuous development of the logistics industry, violent sorting has frequently occurred in the process of sorting items. In related technologies, the sorting process is usually captured by video, and object detection is performed on each static image frame in the video, and based on the object detection results, it is determined whether it is "violent sorting behavior." In the process of determining violent sorting behavior, only object detection is performed on each static image frame. There is useless data in the object detection results, and it is not accurate enough, which can easily lead to many moving packages being missed and "violent sorting behavior" being missed. Therefore, how to accurately determine the target object required in each frame of the image is very important for accurately detecting violent sorting behavior. Summary of the invention

[0003] The present application proposes a method, device, electronic device and storage medium for determining a moving object in an image.

[0004] The first aspect of the present application proposes a method for determining a moving object in an image, the method comprising: determining a plurality of first candidate frames of a target object in a current frame video image, wherein the first candidate frames are used to mark the image area occupied by the target object in the current frame image; determining a motion foreground image of the current frame video image; determining a second candidate frame of the moving target object in the current frame video image based on the motion foreground image; filtering the plurality of first candidate frames based on the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixels in the third candidate frame is greater than a first preset threshold; and determining a target frame of the moving target object based on the second candidate frame and the third candidate frame.

[0005] In one embodiment of the present application, determining the target frame of the moving target object based on the second candidate frame and the third candidate frame includes: determining the overlapping area between the second candidate frame and the third candidate frame; and when the area of ​​the overlapping area is greater than a second preset threshold, using the second candidate frame or the third candidate frame as the target frame of the moving target object.

[0006] In one embodiment of the present application, the method further includes: when the area of ​​the overlapping region is less than or equal to a second preset threshold, using both the second candidate frame and the third candidate frame as target frames of the moving target object.

[0007] In one embodiment of the present application, there are multiple motion foreground images, and determining the second candidate frame of the moving target object in the current frame video image based on the motion foreground images includes: determining the connected area of ​​each of the motion foreground images, and calculating the total area of ​​the connected area in each of the motion foreground images; determining the maximum circumscribed rectangular frame of each of the motion foreground images; obtaining, from the multiple motion foreground images, a target motion foreground image in which the ratio of the total area of ​​the connected area to the area of ​​the maximum circumscribed rectangular frame is greater than a third preset threshold; and determining the maximum circumscribed rectangular frame corresponding to the target motion foreground image as the second candidate frame of the moving target object.

[0008] In one embodiment of the present application, determining the motion foreground image of the current frame video image includes: acquiring a previous frame video image of the current frame video image; determining a background image of the current frame video image based on the previous frame video image; performing foreground recognition on the current frame video image based on the background image to obtain the motion foreground image of the current frame video image, wherein the motion foreground image is used to characterize the change of the current frame video image relative to the background image.

[0009] The present application proposes a method for determining a moving object in an image. In the process of determining a target frame of a moving target object in a current frame video image, multiple first candidate frames of the target object in the current frame video image are determined, and a moving foreground image of the current frame video image is determined, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and the multiple first candidate frames are filtered to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, which have an impact on determining the target frame corresponding to the moving target object in the current frame video image, and the target frame of the moving target object in the current frame video image is accurately determined while improving the computational efficiency.

[0010] A second aspect of the present application proposes a device for determining a moving object in an image, the device comprising: a first determination module, used to determine a plurality of first candidate frames of a target object in a current frame video image, wherein the first candidate frame is used to mark the image area occupied by the target object in the current frame image; a second determination module, used to determine a motion foreground image of the current frame video image; a third determination module, used to determine a second candidate frame of the moving target object in the current frame video image based on the motion foreground image; a filtering module, used to filter the plurality of first candidate frames based on the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixels in the third candidate frame is greater than a first preset threshold; and a fourth determination module, used to determine a target frame of the moving target object based on the second candidate frame and the third candidate frame.

[0011] In one embodiment of the present application, the fourth determination module includes: a determination unit, used to determine the overlapping area between the second candidate box and the third candidate box; a first generation unit, used to use the second candidate box or the third candidate box as the target box of the moving target object when the area of ​​the overlapping area is greater than a second preset threshold.

[0012] In one embodiment of the present application, the fourth determination module also includes: a second generation unit, which is used to use the second candidate box and the third candidate box as target boxes of the moving target object when the area of ​​the overlapping area is less than or equal to a second preset threshold.

[0013] In one embodiment of the present application, there are multiple motion foreground images, and the second determination module is specifically used to: determine the connected area of ​​each of the motion foreground images, and calculate the total area of ​​the connected area in each of the motion foreground images; determine the maximum circumscribed rectangular frame of each of the motion foreground images; obtain a target motion foreground image from the multiple motion foreground images, in which the ratio of the total area of ​​the connected area to the area of ​​the maximum circumscribed rectangular frame is greater than a third preset threshold; and determine the maximum circumscribed rectangular frame corresponding to the target motion foreground image as the second candidate frame of the moving target object.

[0014] In one embodiment of the present application, the first determination module is specifically used to: obtain a previous frame of video image of the current frame of video image; determine a background image of the current frame of video image based on the previous frame of video image; perform foreground recognition on the current frame of video image based on the background image to obtain a motion foreground image of the current frame of video image, wherein the motion foreground image is used to characterize the change of the current frame of video image relative to the background image.

[0015] The present application proposes a device for determining a moving object in an image. In the process of determining a target frame of a moving target object in a current frame video image, a plurality of first candidate frames of the target object in the current frame video image are determined, and a moving foreground image of the current frame video image is determined, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and the plurality of first candidate frames are filtered to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, which have an impact on determining the target frame corresponding to the moving target object in the current frame video image, and the target frame of the moving target object in the current frame video image is accurately determined while improving the computational efficiency.

[0016] The third aspect embodiment of the present application proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for determining a moving object in an image in the embodiment of the present application is implemented.

[0017] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a method for determining a moving object in an image in the embodiment of the present application is provided.

[0018] Other effects of the above optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a method for determining a moving object in an image provided by an embodiment of the present application;

[0020] Figure 2 is a flow chart of another method for determining a moving object in an image provided by an embodiment of the present application;

[0021] Figure 3 is a flow chart of another method for determining a moving object in an image provided by an embodiment of the present application;

[0022] Figure 4 is an example diagram of a moving foreground image provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of a device for determining a moving object in an image provided by an embodiment of the present application;

[0024] Figure 6 is a schematic structural diagram of another device for determining a moving object in an image provided by an embodiment of the present application;

[0025] Figure 7 It is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0027] The following describes a method, device, and electronic device for determining a moving object in an image according to an embodiment of the present application with reference to the accompanying drawings.

[0028] Figure 1 It is a flow chart of a method for determining a moving object in an image provided by an embodiment of the present application. It should be noted that the execution subject of the method for determining a moving object in an image provided by this embodiment is a device for determining a moving object in an image, which can be implemented by software and / or hardware. The device for determining a moving object in an image in this embodiment can be configured in an electronic device, and the electronic device in this embodiment can include a server. This embodiment does not specifically limit the electronic device.

[0029] Figure 1 It is a flowchart of a method for determining a moving object in an image provided in an embodiment of the present application.

[0030] like Figure 1 As shown, the method for determining the moving object in the image may include:

[0031] Step 101 : determining a plurality of first candidate frames of a target object in a current frame of video image, wherein the first candidate frames are used to mark an image area occupied by the target object in the current frame of image.

[0032] In some embodiments, when "violent sorting" is identified in a logistics scenario, in order to detect whether it is in "violent sorting", a video of the process of sorting packages can be captured, and each frame of the captured video can be processed using the method proposed in this embodiment. It should be noted that the above-mentioned target object can be a package object.

[0033] In some embodiments, the above-mentioned determination of multiple first candidate frames of the target object in the current frame video image can be implemented in multiple ways, which are exemplarily described as follows:

[0034] As an exemplary implementation, the current frame video image may be input into a pre-trained neural network model to obtain multiple first candidate frames of the target object in the current frame video image through the neural network model.

[0035] As another exemplary implementation, the current frame video image may be processed based on the features of the target object to obtain a region matching the features of the target object, and a candidate frame of the target object may be determined based on the region.

[0036] Step 102, determining the moving foreground image of the current frame video image.

[0037] The determination of the motion foreground image of the current frame video image may be achieved in a variety of ways, such as a frame difference method, a background modeling method, and an optical flow method, but is not limited thereto.

[0038] Step 103: Determine a second candidate frame of the moving target object in the current frame video image according to the moving foreground image.

[0039] In some embodiments, during violent sorting, when the current package is a moving package, since the package is in a high-speed moving state, motion blur will occur. In order to avoid the inability to accurately detect the candidate frame of the moving target object based on one frame of video image, in this embodiment, the motion foreground image of the current frame video image of the package and the video image of the current frame can be obtained, and segmented according to the image change information, and then the segmented area calculation result is used to determine the second candidate frame of the moving target object in the current frame video image, so as to accurately determine the second candidate frame of the moving target object in the current frame video image based on the motion foreground image.

[0040] Step 104 , filtering the plurality of first candidate frames according to the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixel points in the third candidate frame is greater than a first preset threshold.

[0041] In some embodiments, one implementation method of filtering multiple first candidate frames according to a motion foreground image to obtain a third candidate frame is to determine, for each first candidate frame, the area of ​​the first candidate frame with a pixel point of 1 inside the first candidate frame according to the motion foreground image, and when the area is less than a first preset threshold, it is determined that there is no moving target object in the first candidate frame, and the first candidate frame is filtered out, so as to filter multiple first candidate frames to obtain a filtered third candidate frame, thereby effectively reducing the useless calculation of a large number of static packages and saving computing resources in the process of detecting violent sorting of packages.

[0042] The first preset threshold may be set in combination with an actual scenario, and this embodiment does not specifically limit this.

[0043] Step 105: Determine a target frame of the moving target object based on the second candidate frame and the third candidate frame.

[0044] In some embodiments, the second candidate box is fused with the third candidate box, and the target box of the moving target object is determined based on the overlapping area of ​​the second candidate box and the third candidate box. That is, by combining the detection results of the static image and the dynamic motion information, a large number of useless stationary objects for "brute force sorting" identification are filtered out, which effectively reduces the amount of calculation and improves the calculation efficiency.

[0045] The present application proposes a method for determining a moving object in an image. In the process of determining a target frame of a moving target object in a current frame video image, multiple first candidate frames of the target object in the current frame video image are determined, and a moving foreground image of the current frame video image is determined, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and the multiple first candidate frames are filtered to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, which have an impact on determining the target frame corresponding to the moving target object in the current frame video image, and the target frame of the moving target object in the current frame video image is accurately determined while improving the computational efficiency.

[0046] Figure 2 It is a flowchart of another method for determining a moving object in an image provided in an embodiment of the present application.

[0047] Step 201 : determining a plurality of first candidate frames of a target object in a current frame of video image, wherein the first candidate frames are used to mark an image area occupied by the target object in the current frame of image.

[0048] It should be noted that, for the specific implementation of step 201, reference may be made to the relevant description in the above embodiment.

[0049] Step 202, determining the moving foreground image of the current frame video image.

[0050] In some embodiments, an implementation method of determining a motion foreground image of a current frame video image is to obtain a previous frame video image of the current frame video image, and determine a background image of the current frame video image based on the previous frame video image, and perform foreground recognition on the current frame video image based on the background image to obtain a motion foreground image of the current frame video image, wherein the motion foreground image is used to characterize the change of the current frame video image relative to the background image, thereby effectively and accurately extracting the moving object as the foreground from the still image.

[0051] Step 203: Determine a second candidate frame of the moving target object in the current frame video image according to the moving foreground image.

[0052] In some embodiments, there are multiple motion foreground images, and one implementation method of determining the second candidate frame of the moving target object in the current frame video image based on the motion foreground images is to determine the connected area of ​​each motion foreground image, calculate the total area of ​​the connected area in each motion foreground image, and determine the maximum circumscribed rectangular frame of each motion foreground image, so as to obtain a target motion foreground image from the multiple motion foreground images, in which the ratio of the total area of ​​the connected area to the area of ​​the maximum circumscribed rectangular frame is greater than a third preset threshold, and determine the maximum circumscribed rectangular frame corresponding to the target motion foreground image as the second candidate frame of the moving target object, thereby accurately determining the second candidate frame of the moving target object in the current frame video image based on the connectivity calculation of the motion foreground image.

[0053] Step 204 , filtering the plurality of first candidate frames according to the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixel points in the third candidate frame is greater than a first preset threshold.

[0054] In some embodiments, the area of ​​the motion foreground pixel in each first candidate frame can be determined based on the motion foreground image, and the area of ​​the motion foreground pixel in the first candidate frame can be compared with a first preset threshold corresponding to the target object. If the area of ​​the motion foreground pixel in the first candidate frame is greater than the first preset threshold, the first candidate frame is used as the third candidate frame.

[0055] Among them, it can be understood that the area of ​​the motion foreground pixel in each first candidate frame is determined according to the motion foreground image. The specific method can be: by obtaining the position information of the first candidate frame in the current frame video image, determining the area of ​​the motion foreground pixel located within the position information from the motion foreground image, so as to obtain the area of ​​the motion foreground pixel in the first candidate frame.

[0056] Step 205: determine the overlapping area between the second candidate box and the third candidate box.

[0057] Step 206: When the area of ​​the overlapping region is greater than the second preset threshold, the second candidate frame or the third candidate frame is used as the target frame of the moving target object.

[0058] In some embodiments, when the area of ​​the overlapping region is less than or equal to a second preset threshold, the second candidate frame and the third candidate frame are both used as target frames of the moving target object, thereby accurately determining the target frame of the target object based on the overlapping area between the second candidate frame and the third candidate frame.

[0059] The present application proposes a method for determining a moving object in an image, which determines multiple first candidate frames of a target object in a current frame video image, and determines a moving foreground image of the current frame video image, so as to determine a second candidate frame of a moving target object in the current frame video image based on the moving foreground image, and filters multiple first candidate frames to obtain a third candidate frame, and when the area of ​​the overlapping region between the second candidate frame and the third candidate frame is greater than a second preset threshold, the second candidate frame or the third candidate frame is used as a target frame of the moving target object. Thus, in the process of detecting the movement of the target object, the target frame of the target object is accurately determined based on the screening of the static image and the dynamic motion information.

[0060] Figure 3 It is a flowchart of another method for determining a moving object in an image provided in an embodiment of the present application.

[0061] Step 301: Acquire a video image sequence.

[0062] In some embodiments, the input video stream or local video file may be decoded and frame extracted to extract an image sequence as input for target object detection.

[0063] Step 302: object detection in a single static image.

[0064] In some embodiments, each static image in the above-collected image sequence is obtained, and target detection is performed to extract the target frame B enclosed in each frame of the image. model (t), it can be understood that the method of extracting the target box wrapped in each frame of the image may include but is not limited to target detection and deep learning neural network model.

[0065] Among them, B model (t) represents the target box result of the static image predicted by the model, t is the frame index of the image sequence, and each element is the coordinates of the upper left corner and lower right corner of the target box (x1, y1, x2, y2).

[0066] Step 303: segmentation of the motion foreground area.

[0067] In some embodiments, the motion foreground area segmentation may adopt, but is not limited to, a frame difference method, a background modeling method, or an optical flow method, and this embodiment does not make any specific limitation thereto.

[0068] Among them, the frame difference method and the optical flow method both use the current frame and the previous frame as input to calculate the image change information, and then perform image binary segmentation on the results of the frame difference method or the optical flow method according to an empirical threshold of the corresponding environment to obtain the moving foreground image, such as Figure 4 As shown, it can be understood that Figure 4 The left side is a static color (red-green-blue, RGB) image. Figure 4 The part on the right excluding the dotted box is the motion foreground segmentation result, and the gray area is the motion foreground image F(x, y, t).

[0069] It can be understood that motion foreground segmentation refers to marking the pixels associated with each independent motion in the sequence feature multiple motions, and clustering these pixels according to the objects they belong to. Its main purpose is to extract the moving object as the foreground from the static background.

[0070] Step 304: Generate additional moving object target frames.

[0071] In some embodiments, according to the motion foreground segmentation result obtained in step 303, Figure 4 For the gray moving foreground image on the right, the connected area is calculated, and the maximum circumscribed rectangular frame is obtained for the outermost edge of each connected area, such as Figure 4 The rectangular boxes c and d on the right, due to the high-speed motion characteristics of the package itself, show that the moving area of ​​the package often presents a connected area with few holes inside in the moving foreground segmentation result map. In addition, in logistics scenarios, most packages are rectangular, so the total area of ​​the connected area S can be calculated roi and the maximum outer frame area S rec ratio = S ri o / S rec , as a constraint for generating additional motion parcel target boxes, when ratio is greater than a certain threshold, the candidate box can be considered as a motion parcel target, such as Figure 4 The candidate box c on the right is the real moving package, while the candidate box d is caused by the local movement of the human arm.

[0072] In addition, the candidate boxes whose total area of ​​connected regions is less than a certain threshold can be filtered out to filter out noise points and improve the accuracy of the detection results.

[0073] Step 305: fusion of results.

[0074] In some embodiments, the stationary target objects in the target detection results of the static image are first filtered out based on the motion segmentation foreground image obtained in step 303, wherein the motion segmentation foreground image obtained in step 303 is a binary image, the pixels in the motion foreground area are 1, and the background pixels are 0.

[0075] The first candidate frame is obtained from the package detection of each static image frame, and the area of ​​the internal pixel is calculated. If it is less than a certain threshold, it can be considered that there is no moving target in the first candidate frame, so as to filter out a large number of static first candidate frames, and then the filtered first candidate frames and the generated additional moving object target frames are merged. For two frames whose overlapping area is greater than a certain threshold, one of them is randomly deleted and only one is retained as the final target candidate frame result. For two frames whose overlapping area is less than or equal to a certain threshold, both are used as the final target candidate frames.

[0076] Step 306: output the moving object detection result.

[0077] The present application proposes a method for determining a moving object in an image, which uses the image sequence of the acquired video as input to perform package detection on each static image frame and perform motion foreground area segmentation, thereby generating an additional motion package target frame, and then fuses the motion package target frame with the detection frame of the static target detection result to obtain a target candidate frame, and outputs a moving object detection result. Thus, in the process of detecting the motion of the target object, the detection result of the static image and the dynamic motion information are fused to accurately obtain the moving object detection result and output it.

[0078] Figure 5 It is a structural schematic diagram of a device for determining a moving object in an image provided in an embodiment of the present application.

[0079] like Figure 5 As shown, the device 500 for determining a moving object in an image includes:

[0080] The first determination module 501 is used to determine a plurality of first candidate frames of a target object in a current frame of video image, wherein the first candidate frames are used to mark an image area occupied by the target object in the current frame of image.

[0081] The second determining module 502 is used to determine the moving foreground image of the current frame video image.

[0082] The third determination module 503 is used to determine a second candidate frame of the moving target object in the current frame video image according to the moving foreground image.

[0083] The filtering module 504 is used to filter the multiple first candidate frames according to the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixel points in the third candidate frame is greater than a first preset threshold.

[0084] The fourth determination module 505 is used to determine a target frame of the moving target object according to the second candidate frame and the third candidate frame.

[0085] The present application proposes a device for determining a moving object in an image. In the process of determining a target frame of a moving target object in a current frame video image, a plurality of first candidate frames of the target object in the current frame video image are determined, and a moving foreground image of the current frame video image is determined, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and the plurality of first candidate frames are filtered to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, which have an impact on determining the target frame corresponding to the moving target object in the current frame video image, and the target frame of the moving target object in the current frame video image is accurately determined while improving the computational efficiency.

[0086] In one embodiment of the present application, Figure 6 , the fourth determining module 505 includes:

[0087] The determining unit 5051 is used to determine the overlapping area between the second candidate box and the third candidate box.

[0088] The first generating unit 5052 is configured to use the second candidate box or the third candidate box as a target box of the moving target object when the area of ​​the overlapping region is greater than a second preset threshold.

[0089] In one embodiment of the present application, Figure 6 , the fourth determining module 505 further includes:

[0090] The second generating unit 5053 is used to use the second candidate frame and the third candidate frame as target frames of the moving target object when the area of ​​the overlapping region is less than or equal to a second preset threshold.

[0091] In one embodiment of the present application, Figure 6 , the second determining module 502 is specifically configured to:

[0092] The connected regions of each motion foreground image are determined, and the total area of ​​the connected regions in each motion foreground image is calculated.

[0093] Determine the maximum bounding rectangle of each moving foreground image.

[0094] A target moving foreground image is obtained from the multiple moving foreground images, wherein the ratio of the total area of ​​the connected regions to the area of ​​the largest circumscribed rectangular frame is greater than a third preset threshold.

[0095] The largest circumscribed rectangular frame corresponding to the target moving foreground image is determined as the second candidate frame of the moving target object.

[0096] In one embodiment of the present application, Figure 6 , the first determining module 501 is specifically used for:

[0097] Get the previous video frame of the current video frame.

[0098] According to the previous frame of video image, a background image of the current frame of video image is determined.

[0099] The foreground of the current frame video image is identified according to the background image to obtain a motion foreground image of the current frame video image, wherein the motion foreground image is used to characterize the change of the current frame video image relative to the background image.

[0100] The present application proposes a method for determining a moving object in an image. In the process of determining a target frame of a moving target object in a current frame video image, multiple first candidate frames of the target object in the current frame video image are determined, and a moving foreground image of the current frame video image is determined, so as to determine a second candidate frame of the moving target object in the current frame video image based on the moving foreground image, and the multiple first candidate frames are filtered to obtain a third candidate frame, so as to determine the target frame of the moving target object based on the second candidate frame and the third candidate frame. Thus, by filtering the candidate frame of the target object in the current frame video image in combination with the moving foreground image of the current frame video image, a large number of useless candidate frames are reduced, which have an impact on determining the target frame corresponding to the moving target object in the current frame video image, and the target frame of the moving target object in the current frame video image is accurately determined while improving the computational efficiency.

[0101] like Figure 7 , which is a block diagram of an electronic device according to an embodiment of the present application.

[0102] like Figure 7 As shown, the electronic device includes:

[0103] Memory 701 , processor 702 , and computer instructions stored in the memory 701 and executable on the processor 702 .

[0104] When the processor 702 executes the instructions, the method for determining a moving object in an image provided in the above embodiment is implemented.

[0105] Furthermore, the electronic device further comprises:

[0106] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0107] The memory 701 is used to store computer instructions that can be executed on the processor 702 .

[0108] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0109] The processor 702 is configured to implement the method for determining a moving object in an image of the above embodiment when executing a program.

[0110] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0111] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0112] The processor 702 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0113] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0114] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0115] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for determining a moving object in an image, characterized in that: The method comprises: Determine a plurality of first candidate frames of a target object in a current frame of video image, wherein the first candidate frames are used to mark an image area occupied by the target object in the current frame of image; Determine the moving foreground image of the current frame video image; Determine, according to the moving foreground image, a second candidate frame of the moving target object in the current frame video image; According to the motion foreground image, filtering the plurality of first candidate frames to obtain a third candidate frame, wherein the area of ​​the motion foreground pixel points in the third candidate frame is greater than a first preset threshold; Determining a target frame of the moving target object according to the second candidate frame and the third candidate frame; There are multiple moving foreground images, the moving foreground images include a target moving foreground image, and the second candidate frame is the largest circumscribed rectangular frame corresponding to the target moving foreground image.

2. The method according to claim 1, characterized in that The step of determining a target frame of the moving target object according to the second candidate frame and the third candidate frame includes: determining an overlapping area between the second candidate frame and the third candidate frame; When the area of ​​the overlapping region is greater than a second preset threshold, the second candidate frame or the third candidate frame is used as the target frame of the moving target object.

3. The method according to claim 2, characterized in that The method further comprises: When the area of ​​the overlapping region is less than or equal to a second preset threshold, both the second candidate frame and the third candidate frame are used as target frames of the moving target object.

4. The method according to claim 2, characterized in that The step of determining a second candidate frame of the moving target object in the current frame video image according to the moving foreground image comprises: Determining the connected regions of each of the motion foreground images, and calculating the total area of ​​the connected regions in each of the motion foreground images; Determine the maximum circumscribed rectangular frame of each of the moving foreground images; From the plurality of motion foreground images, obtaining a target motion foreground image whose ratio of the total area of ​​the connected regions to the area of ​​the largest circumscribed rectangular frame is greater than a third preset threshold; The largest circumscribed rectangular frame corresponding to the target moving foreground image is determined as a second candidate frame of the moving target object.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining the moving foreground image of the current frame video image comprises: Obtaining a previous frame of video image of the current frame of video image; Determine the background image of the current frame of video image according to the previous frame of video image; Foreground recognition is performed on the current frame video image according to the background image to obtain a motion foreground image of the current frame video image, wherein the motion foreground image is used to characterize changes of the current frame video image relative to the background image.

6. A device for determining a moving object in an image, characterized in that: The device comprises: A first determination module is used to determine a plurality of first candidate frames of a target object in a current frame video image, wherein the first candidate frames are used to mark an image area occupied by the target object in the current frame image; A second determining module is used to determine the motion foreground image of the current frame video image; A third determination module is used to determine a second candidate frame of the moving target object in the current frame video image according to the moving foreground image; A filtering module, configured to filter the plurality of first candidate frames according to the motion foreground image to obtain a third candidate frame, wherein the area of ​​the motion foreground pixel points in the third candidate frame is greater than a first preset threshold; a fourth determining module, configured to determine a target frame of the moving target object according to the second candidate frame and the third candidate frame; There are multiple moving foreground images, the moving foreground images include a target moving foreground image, and the second candidate frame is the largest circumscribed rectangular frame corresponding to the target moving foreground image.

7. The device according to claim 6, characterized in that The fourth determining module includes: a determining unit, configured to determine an overlapping area between the second candidate frame and the third candidate frame; The first generating unit is configured to use the second candidate frame or the third candidate frame as a target frame of the moving target object when the area of ​​the overlapping region is greater than a second preset threshold.

8. The device according to claim 7, characterized in that The fourth determination module further includes: The second generating unit is configured to use the second candidate frame and the third candidate frame as target frames of the moving target object when the area of ​​the overlapping region is less than or equal to a second preset threshold.

9. The device according to claim 7, characterized in that There are multiple moving foreground images, and the second determining module is specifically used to: Determining the connected regions of each of the motion foreground images, and calculating the total area of ​​the connected regions in each of the motion foreground images; Determine the maximum circumscribed rectangular frame of each of the moving foreground images; From the plurality of motion foreground images, obtaining a target motion foreground image whose ratio of the total area of ​​the connected regions to the area of ​​the largest circumscribed rectangular frame is greater than a third preset threshold; The largest circumscribed rectangular frame corresponding to the target moving foreground image is determined as a second candidate frame of the moving target object.

10. The device according to any one of claims 6 to 9, characterized in that: The first determining module is specifically configured to: Obtaining a previous frame of video image of the current frame of video image; Determine the background image of the current frame of video image according to the previous frame of video image; Foreground recognition is performed on the current frame video image according to the background image to obtain a motion foreground image of the current frame video image, wherein the motion foreground image is used to characterize changes of the current frame video image relative to the background image.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining a moving object in an image as described in any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for determining a moving object in an image as described in any one of claims 1 to 5 is implemented.

Images