Target detection method, apparatus and device, and readable medium

Abstract

The embodiment of the invention provides a target detection method and device, equipment and a readable medium. The method comprises the following steps: acquiring a target video for a stacking area of an object; determining the displacement of each image block in the stacking area in the vertical direction based on any two adjacent frames of the target video; and in response to determining that the displacement of one or more of the image blocks in the vertical direction exceeds a predetermined displacement threshold, determining one or more target areas in the stacking area based on the one or more image blocks, the target areas indicating the falling area of the object. In this way, the manual annotation cost of the object can be reduced, and the generalization ability of the target detection method is improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer vision, and more particularly, to object detection methods, apparatuses, devices, and computer-readable storage media. Background Art

[0002] With the development of computer technology, image processing technology has also developed. Through computer vision and image processing methods, target objects in videos or image sequences can be identified, tracked, and located. However, when the number of target objects is large and the types are diverse, the distribution of the visual features of the target objects varies greatly. It is necessary to manually label various types of target objects to achieve the recognition, tracking, and location of the target objects, which leads to problems such as high labeling costs and weak generalization ability of target detection algorithms. Summary of the invention

[0003] In a first aspect of the present disclosure, a target detection method is provided, comprising: acquiring a target video of a stacking area of ​​an object; determining, based on any two adjacent frames of the target video, a displacement size of each image block in the stacking area in a vertical direction; in response to determining that a displacement size of one or more of the image blocks in the vertical direction exceeds a predetermined displacement threshold, determining, based on the one or more image blocks, one or more target areas in the stacking area, the target areas indicating areas where the object falls.

[0004] In a second aspect of the present disclosure, a target detection device is provided, comprising: an acquisition module, configured to acquire a target video for a stacking area of ​​an object; a first determination module, configured to determine, based on any two adjacent frames of the target video, the displacement size of each image block in the stacking area; and a second determination module, configured to determine, in response to determining that the displacement size of one or more image blocks in the vertical direction among the image blocks exceeds a predetermined displacement threshold, one or more target areas in the stacking area based on the one or more image blocks, the target area indicating the area where the object falls.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect of the present disclosure.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to perform the method according to the first aspect of the present disclosure.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the instructions are executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the contents described in the summary of the invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart showing a process of target detection according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic diagram showing a stacking area according to some embodiments of the present disclosure;

[0013] FIG. 4A to FIG. 4C A schematic diagram showing a target area according to some embodiments of the present disclosure;

[0014] Figure 5 A schematic diagram showing a first abnormality level and a second abnormality level according to some embodiments of the present disclosure;

[0015] Figure 6 A block diagram showing an apparatus for target detection according to some embodiments of the present disclosure; and

[0016] Figure 7 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0019] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network", "neural network" or "network", and these terms are used interchangeably in this article.

[0020] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes input from the previous layer.

[0021] Generally, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association from input to output (also called the mapping of input to output) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values ​​obtained from the training to determine the corresponding output.

[0022] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0023] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0024] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.

[0025] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] As briefly mentioned above, computer vision and image processing methods can be used to identify, track, and locate target objects based on video or image sequences. However, when there are a large number of target objects and a wide variety of types, the distribution of target object category visual features varies greatly, and various types of target objects need to be manually labeled to achieve target object identification, tracking, and location, which leads to problems such as high labeling costs and weak generalization ability of target detection algorithms.

[0027] In some scenarios, when loading and unloading personnel are stacking goods on or unloading goods from transport vehicles, improper operation of the loading and unloading personnel (for example, not placing goods in order according to their size, weight, fragility, etc., or not following the distribution of cargo controls and unloading goods by brute force, etc.) may cause the center of gravity of the goods to be unbalanced and collapse and fall, causing damage to the goods. Usually, an object detection algorithm (for example, a convolutional neural network model) can be used to identify whether the goods have fallen. However, the goods need to be manually labeled to train the object detection algorithm. Due to the large number of goods and the variety of outer packaging types (for example, long strips, balls, rings, or irregular geometric shapes, etc.), the category visual feature distribution of the outer packaging of the goods varies greatly, and various types of outer packaging need to be manually labeled one by one to train the object detection algorithm, resulting in high labeling costs and weak generalization ability of the object detection algorithm.

[0028] The embodiments of the present disclosure provide a target detection scheme. According to the various embodiments of the present disclosure, a target video for the stacking area of ​​an object is obtained. Based on any two adjacent frames of the target video, the displacement size of each image block in the stacking area is determined. In response to determining that the displacement size of one or more image blocks in the various image blocks in the vertical direction exceeds a predetermined displacement threshold, one or more target areas are determined in the stacking area based on one or more image blocks, and the target area indicates the area where the object falls. In this way, it is possible to determine whether an image block falls based on the displacement size of the image block in the vertical direction and the predetermined displacement threshold, and determine one or more target areas indicating that the object falls in the stacking area based on one or more image blocks. Compared with the above-mentioned target detection algorithm, the target detection method provided by the embodiments of the present disclosure does not require manual labeling, reduces labor costs, and improves the generalization ability of the target detection method.

[0029] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Environment 100 may include acquisition devices 110-1, 110-2, ..., 110-N and a display device 120. For ease of discussion, acquisition devices 110-1, 110-2, ..., 110-N may be collectively or individually referred to as acquisition devices 110.

[0030] In the environment 100, the acquisition device 110 may include any device for acquiring video data, such as an image sensor, an infrared sensor, etc. The display device 120 may include any device for displaying video data and having computing capabilities (such as video image processing). The acquisition device 110 and the display device 120 are in communication connection, and the acquisition device 110 may transmit the acquired video data to the display device 120 for display and / or processing.

[0031] The communication connection between the acquisition device 110 and the display device 120 can be established in a wired manner or a wireless manner. The communication connection may include but is not limited to a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect.

[0032] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0033] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0034] Figure 2A flowchart of a process 200 of object detection according to some embodiments of the present disclosure is shown. The process 200 of object detection performed by a display device will be taken as an example for description.

[0035] In block 210 , the display device 120 acquires a target video of a stacking area of ​​an object from the acquisition device 110 .

[0036] The target video may be acquired by the acquisition device 110 and transmitted to the display device 120, or the acquisition device 110 may pre-process the acquired video and then transmit it to the display device 120. For example, the acquisition device 110 may pre-process the acquired video by dividing the video into segments of a predetermined length, adjusting the video resolution, and the like.

[0037] The object may be an object with a certain size and / or shape. For example, the object may be a cargo, an outer packaging of the cargo, etc. For example, the shape of the object may be a strip, a sphere, a ring, an irregular geometric shape, etc. Multiple objects may be stacked in the stacking area in any manner, for example, stacked according to the size and / or shape of the object.

[0038] In some embodiments, the acquisition device 110 can acquire video data of the working area and transmit it to the display device 120. The working area can be, for example, a warehouse or a transportation vehicle (e.g., a vehicle). The working area can accommodate objects that are being palletized or unpalletized, or can accommodate objects that have been palletized. The stacking area of ​​the objects is included in the working area.

[0039] For example, see Figure 3 , the figure shows a picture of the interior of a truck compartment, in which a plurality of boxes are stacked. In this example, the interior of the truck compartment is a working area, and the area where the plurality of boxes are stacked is a stacking area, that is, the area framed by lines in the figure is the stacking area.

[0040] In block 220 , the display device 120 determines the displacement size of each image block in the stacking area in the vertical direction based on any two adjacent frames of the target video.

[0041] In some embodiments, the display device 120 may determine the displacement of each image block in the stacking area in the vertical direction based on any two adjacent frames in all frames in the target video. In other embodiments, the display device 120 may also extract any multiple frames from the target video, and determine the displacement of each image block in the stacking area in the vertical direction based on any two adjacent frames in any multiple frames.

[0042] An image block may be formed by one or more pixels. For example, 1×1 pixels may constitute an image block, 2×2 pixels may constitute an image block, and 4×4 pixels may constitute an image block, which is not limited in the embodiments of the present disclosure.

[0043] In some embodiments, before determining the displacement size of each image block in the stacking area in the vertical direction, image segmentation may be performed on all frames in the target video or any multiple frames extracted to obtain images of the stacking area of ​​each of the multiple frames.

[0044] For example, the display device 120 may use a trained segmentation model to perform image segmentation on multiple frames in the target video to obtain images of the stacked regions of the multiple frames. For example, the segmentation model may be a real-time semantic segmentation target detection model (YOLO Semantic Segmentation). For example, the real-time semantic segmentation target detection model may be trained using images labeled with stacked regions.

[0045] Continue to see Figure 3 The display device 120 can segment the image of the stacking area (the area framed by the image lines) using the trained segmentation model. It should be understood that in order to further reduce the amount of calculation, the stacking area can be roughly segmented, and only the approximate stacking area needs to be segmented.

[0046] After obtaining the image of the stacked area, the display device 120 can determine the displacement size of each image block in the stacked area in the vertical direction based on the stacked areas of any two adjacent frames in the target video. For example, the display device 120 can calculate the displacement size of the image block between any two adjacent frames based on an optical flow algorithm or a distribution consistency algorithm. The displacement size of the image block may include, for example, the displacement size in the horizontal direction and the displacement size in the vertical direction, where the horizontal direction and the vertical direction are relative to the ground. The display device 120 can determine the displacement of each image block in the vertical direction based on the displacement vector of each image block in the stacked area.

[0047] Taking the optical flow algorithm as an example, the display device 120 can use the optical flow algorithm to calculate the optical flow graph I of the stacking area of ​​any two adjacent frames in the target video. ij (d x ,d y ), through the vertical displacement vector d y To determine the vertical displacement of the image block.

[0048] In some embodiments, the display device 120 may not perform image segmentation on multiple frames in the target video, but directly use the above method to determine the displacement size of each image block in the stacking area in the vertical direction.

[0049] Continue to see Figure 2 In box 230, in response to determining that the displacement size of one or more image blocks in the vertical direction of the individual image blocks exceeds a predetermined displacement threshold, the display device 120 determines one or more target areas in the stacking area based on the one or more image blocks, and the target area indicates the area where the object falls.

[0050] In some embodiments, the display device 120 may cluster one or more image blocks to determine one or more target areas. For example, image blocks with consistent motion behavior (e.g., displacement in the vertical direction exceeding a predetermined displacement threshold) may be clustered, and one or more areas may be obtained after clustering, each of which may include one or more image blocks. The display device 120 may perform contour extraction on the clustered image blocks to determine one or more areas in the stacked area indicating that the object has fallen as the target area.

[0051] For example, see FIG. 4A to FIG. 4C , the arrow in the figure indicates the moving direction of the image block, and the contour indicates the target area. It should be understood that the clustering method and the contour extraction method can adopt any method in the relevant technology, and the embodiment of the present disclosure is not limited thereto.

[0052] Furthermore, the display device 120 may determine a first abnormality level based on the displacement size and distance threshold of each image block in the vertical direction, and may also determine a second abnormality level based on a proportion of the area of ​​one or more target areas in the stacking area and a proportion threshold.

[0053] In some embodiments, the display device 120 can classify the first abnormality level by the distance threshold. Specifically, the first abnormality level can be divided into mild, moderate, severe and other levels from small to large by the distance threshold. For example, 0.1m corresponds to mild, 0.3m corresponds to moderate, and 0.5m corresponds to severe, that is, if the displacement of the image block in the vertical direction exceeds 0.1m but does not exceed 0.3m, the first abnormality level is mild; if the displacement of the image block in the vertical direction exceeds 0.3m but does not exceed 0.5m, the first abnormality level is moderate; if the displacement of the image block in the vertical direction exceeds 0.5m, the first abnormality level is severe.

[0054] In some embodiments, the display device 120 can classify the second abnormality level by the proportion threshold. Specifically, the second abnormality level can be divided into mild, moderate, severe, etc. from small to large by the proportion threshold. For example, 0.1m 2 Corresponds to mild, 0.3m 2 Corresponds to moderate, 0.5m 2 Corresponding to heavy, that is, the proportion of image blocks in the stacking area exceeds 0.1m2 Not more than 0.3m 2 , the second abnormality level is mild; the image block accounts for more than 0.3m in the stacking area 2 Not more than 0.5m 2 , the second abnormality level is moderate; the image block accounts for more than 0.5m in the stacking area 2 , the second abnormality level is severe.

[0055] In some embodiments, after determining the first abnormality level and / or the second abnormality level of the image block, the display device 120 may present the first abnormality level and / or the second abnormality level in association with the target video.

[0056] For example, the display device 120 may present the first abnormality level and / or the second abnormality level in one or more frames in the target video, or insert one or more frames into the target video, wherein the inserted one or more frames include the first abnormality level and / or the second abnormality level, so that the display device 120 presents the first abnormality level and / or the second abnormality level simultaneously during the process of presenting the target video.

[0057] For example, see Figure 5 The two intersecting arrows in the figure can represent the reference coordinate system, the downward arrow can represent the first abnormality level, and the length of the arrow can represent the severity of the abnormality. For example, the longer the downward arrow is, the higher the abnormality level is; the area framed by lines in the figure can represent the second abnormality level, and the size of the area framed by lines can represent the severity of the abnormality. For example, the larger the area, the higher the abnormality level is.

[0058] In some embodiments, the display device 120 may present one or more target areas in association with the target video. For example, the display device 120 may present one or more target areas in one or more frames of the target video, or insert one or more frames into the target video, wherein the inserted one or more frames include one or more target areas, so that the display device 120 presents the target video while presenting the one or more target areas. FIG. 4A to FIG. 4C ,The contour in the figure can represent the target area.

[0059] Further, the display device 120 may filter the target area based on the predetermined area of ​​the object and the area of ​​the target area. For example, the predetermined area of ​​the object may include the maximum area or the minimum area of ​​the areas of all the objects in the stacking area. If the area of ​​the target area is smaller than the above minimum area, it means that the target area is a noise area and needs to be filtered out; if the area of ​​the target area is larger than the above maximum area, it means that the target area includes one or more objects and does not need to be filtered.

[0060] According to an embodiment of the present disclosure, a target video of a stacking area for an object is obtained. Based on any two adjacent frames of the target video, the displacement size of each image block in the stacking area in the vertical direction is determined. In response to determining that the displacement size of one or more image blocks in each image block exceeds a predetermined displacement threshold, one or more target areas are determined in the stacking area based on one or more image blocks, and the target area indicates the area where the object has fallen. In this way, it is possible to determine whether an image block has fallen based on the displacement size of the image block in the vertical direction and a predetermined displacement threshold, and determine one or more target areas indicating that the object has fallen in the stacking area based on one or more image blocks. Compared with the above-mentioned target detection algorithm, the target detection method provided by the embodiment of the present disclosure does not require manual labeling, reduces labor costs, and improves the generalization ability of the target detection method.

[0061] Figure 6 FIG. 6 is a block diagram of a target detection apparatus 600 according to some embodiments of the present disclosure. The apparatus 600 may be implemented as or include the display device 120 .

[0062] The device 600 includes an acquisition module 610 configured to acquire a target video of a stacking area of ​​an object. The device 600 also includes a first determination module 620 configured to determine the displacement size of each image block in the stacking area in the vertical direction based on any two adjacent frames of the target video. The device 600 also includes a second determination module 630 configured to determine one or more target areas in the stacking area based on the one or more image blocks in response to determining that the displacement size of one or more image blocks in the vertical direction of the image blocks exceeds a predetermined displacement threshold, wherein the target area indicates the area where the object falls.

[0063] In some embodiments, the first determination module 620 is further configured to cluster the one or more image blocks; and perform contour extraction on the clustered image blocks to determine the one or more target areas in the stacked area.

[0064] In some embodiments, the second determination module 630 is further configured to perform image segmentation on any two adjacent frames of the target video to obtain the stacking areas of each of the two adjacent frames; and based on the stacking areas of each of the two adjacent frames, determine the vertical displacement of each image block in the stacking area.

[0065] In some embodiments, the device 600 also includes a third determination module, which is configured to determine a first abnormality level based on the size of the displacement of each image block in the vertical direction and a distance threshold; and determine a second abnormality level based on the proportion of the area of ​​the one or more target areas in the stacking area and the proportion threshold.

[0066] In some embodiments, the apparatus 600 further includes a first presentation module configured to present the first abnormality level and / or the second abnormality level in association with the target video.

[0067] In some embodiments, the apparatus 600 further includes a second presentation module configured to present the one or more target areas in association with the target video.

[0068] In some embodiments, the apparatus 600 further comprises a filtering module configured to filter the target area based on a predetermined area of ​​the object and an area of ​​the target area.

[0069] Figure 7 1 shows a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0070] like Figure 7 As shown, the electronic device 700 is in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of the electronic device 700.

[0071] The electronic device 700 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 700.

[0072] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0073] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0074] The input device 750 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0075] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0076] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0077] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0078] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0079] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0080] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the implementations disclosed herein.

Claims

1. A target detection method, comprising: Get a target video for the stacked area of ​​the object; Based on any two adjacent frames of the target video, determining the displacement size of each image block in the stacking area in the vertical direction; as well as In response to determining that the displacement size of one or more image blocks in the individual image blocks in the vertical direction exceeds a predetermined displacement threshold, one or more target areas are determined in the stacking area based on the one or more image blocks, and the target areas indicate areas where the object falls.

2. The method according to claim 1, wherein determining one or more target areas in the stacked area based on the one or more image blocks comprises: clustering the one or more image blocks; as well as Contour extraction is performed on the clustered image blocks to determine the one or more target areas in the stacking area.

3. The method according to claim 1, wherein determining the displacement size of each image block in the stacking area in the vertical direction based on any two adjacent frames of the target video comprises: Performing image segmentation on any two adjacent frames of the target video to obtain the stacking regions of the two adjacent frames; as well as Based on the stacking areas of the two adjacent frames, the displacement of each image block in the stacking area in the vertical direction is determined.

4. The method according to claim 1, further comprising: Determining a first abnormality level based on the size and distance threshold of the displacement of each image block in the vertical direction; as well as A second abnormality level is determined based on a proportion of the area of ​​the one or more target areas to the stacking area and a proportion threshold.

5. The method according to claim 4, further comprising: The first anomaly level and / or the second anomaly level are presented in association with the target video.

6. The method according to claim 1, further comprising: The one or more target areas are presented in association with the target video.

7. The method according to claim 1, further comprising: The target area is filtered based on the predetermined area of ​​the object and the area of ​​the target area.

8. A target detection device, comprising: An acquisition module configured to acquire a target video of a stacking area of ​​an object; A first determination module is configured to determine the displacement size of each image block in the stacking area in the vertical direction based on any two adjacent frames of the target video; as well as The second determination module is configured to determine one or more target areas in the stacking area based on the one or more image blocks in response to determining that the displacement size of one or more image blocks in the vertical direction among the individual image blocks exceeds a predetermined displacement threshold, and the target area indicates the area where the object falls.

9. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 7.

Images