Image detection method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose an image detection method, device, electronic device and computer-readable medium. A specific implementation of the method includes: based on the feature map of the image to be detected, generating a detection frame and label corresponding to the target object in the image to be detected, the label including the position information and category information of the target object; based on the category information, determining whether the image to be detected displays a first object of a first category and a second object of a second category; in response to determining that the image to be detected displays a first object and a second object, based on the position information corresponding to the first object and the second object respectively, determining the overlap rate of the detection frames corresponding to the first object and the second object respectively; based on the overlap rate, generating image detection information of the image to be detected, the image detection information is used to characterize whether the first object is placed on the second object in three-dimensional space. This implementation achieves improved detection efficiency.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an image detection method, device, electronic device, and computer-readable medium. Background Art

[0002] With the rapid development of e-commerce, the logistics business has also flourished. At the same time, higher requirements have been put forward for the monitoring of the status of package transportation and distribution. For example, in the terminal logistics link, it is generally necessary to temporarily store the package in the business department (also known as the outlet, etc.), and then the courier will select the package to be delivered and load it to the customer. In this process, in order to avoid damage to the package, it is not allowed to be placed directly on the ground, and it needs to be placed in a designated location such as a package storage basket or a mobile pallet. At present, the monitoring of this kind of link is generally carried out through manual spot checks, and special personnel are arranged to regularly or randomly check the monitoring video.

[0003] However, due to the large number of business departments and cameras, manual spot checks are time-consuming and labor-intensive and are difficult to meet actual needs. Summary of the invention

[0004] The content section of this disclosure is used to introduce concepts in a brief form, and these concepts will be described in detail in the specific implementation section below. The content section of this disclosure is not intended to identify the key features or essential features of the technical solution for which protection is sought, nor is it intended to limit the scope of the technical solution for which protection is sought. Some embodiments of the present disclosure propose image detection methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present disclosure provide an image detection method, the method comprising: generating a detection frame and a label corresponding to a target object in the image to be detected based on a feature map of the image to be detected, the label comprising position information and category information of the target object; determining whether the image to be detected displays a first object of a first category and a second object of a second category based on the category information; in response to determining that the image to be detected displays a first object and a second object, determining an overlap rate of detection frames corresponding to the first object and the second object respectively based on position information corresponding to the first object and the second object respectively; generating image detection information of the image to be detected based on the overlap rate, the image detection information being used to characterize whether the first object is placed on the second object in three-dimensional space.

[0006] In a second aspect, some embodiments of the present disclosure provide an image detection device, comprising: a detection frame generation unit, configured to generate a detection frame and a label corresponding to a target object in the image to be detected based on a feature map of the image to be detected, the label comprising position information and category information of the target object; an object determination unit, configured to determine whether the image to be detected displays a first object of a first category and a second object of a second category based on the category information; an overlap rate determination unit, configured to determine the overlap rate of detection frames corresponding to the first object and the second object respectively based on position information corresponding to the first object and the second object respectively in response to determining that the image to be detected displays a first object and a second object; and an image detection information generation unit, configured to generate image detection information of the image to be detected based on the overlap rate, the image detection information being used to characterize whether the first object is placed on the second object in three-dimensional space.

[0007] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0008] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0009] The above-mentioned embodiments of the present disclosure have the following beneficial effects: automatic detection of whether the first object (e.g., a package) is placed on the second object (e.g., a trailer) is realized, and the detection efficiency is improved by avoiding manual detection. In this process, detection frames of objects of different categories are generated, and the positional relationship in the three-dimensional space is determined by the detection frames. Thus, the positional relationship is determined by a single image. Compared with the related art, it does not require the combination of multiple images, simplifies the determination process, and further improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0011] Figure 1 is a schematic diagram of an application scenario of the image detection method according to some embodiments of the present disclosure;

[0012] Figure 2 is a flow chart of some embodiments of the image detection method according to the present disclosure;

[0013] Figure 3 are flow charts of other embodiments of the image detection method according to the present disclosure;

[0014] Figure 4 is a schematic structural diagram of some embodiments of the image detection device according to the present disclosure;

[0015] Figure 5 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Figure 1 It is a schematic diagram of an application scenario of the image detection method of some embodiments of the present disclosure.

[0023] exist Figure 1In the application scenario, first, the computing device 101 can first generate a detection frame (such as detection frame 1041 and detection frame 1042 in the figure) and a label corresponding to the target object in the image 102 to be detected based on the image 102 to be detected and the feature map 103 using the target detection technology. Among them, the target object can be different according to different actual needs. In this application scenario, the target object can be of two categories, one is a package, and the other is a package placement device (such as a cart). The label of the target object includes the location information and category information of the target object. For example, the detection frame corresponding to the placement device is the detection frame 1041, and the label is (3, 3, 10, 10,) as shown in the mark 1043. Among them, "3, 3, 10, 10" represents the position and width and height of the detection frame 1041, and "cart" is the category information. Based on the category information, it is determined whether the image 102 to be detected shows a first object of the first category and a second object of the second category. In this application scenario, the first category can be a package, and the second category can be a package placement device. Since the image to be detected 102 contains both types of objects, the computing device 101 can determine the overlap rate 105 of the detection frames corresponding to the first object and the second object respectively based on the position information corresponding to the first object and the second object respectively. As can be seen from the figure, the detection frame 1042 is completely contained in the detection frame 1041, so the overlap rate is 100%. At this time, it can be considered that the first object is placed on the second object in three-dimensional space. Therefore, image detection information 106 of the image to be detected 102 can be generated, and the image detection information 106 is used to characterize the placement of the package on the cart.

[0024] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

[0025] Continue to refer Figure 2 , shows a process 200 of some embodiments of the image detection method according to the present disclosure. The image detection method comprises the following steps:

[0026] Step 201: Based on the feature map of the image to be detected, a detection frame and a label corresponding to the target object in the image to be detected are generated, and the label includes the location information and category information of the target object.

[0027] In some embodiments, the execution subject of the image detection method (for example Figure 1Electronic devices (electronic devices in the image to be detected) can generate detection boxes and labels corresponding to target objects in the image to be detected based on the feature map of the image to be detected, using a variety of target detection algorithms and classification algorithms, such as the SSD (Single Shot MultiBoxDetector) algorithm, support vector machines (SVM), etc. Among them, the target object may be different according to actual needs. As an example, in the logistics industry, when it is necessary to determine whether a package is placed in a designated package placement device, the target object may be a package and a package placement device (such as a trailer, a storage basket, etc.). Furthermore, since it is not necessary to identify the placement position of all packages, the target object may be a soft package (such as a package in a file bag, etc.). In addition, the feature map (Feature Map) of the image to be detected can be used to characterize various features of the image. The feature map of the image to be detected can be obtained by various feature extraction algorithms, such as the SIFT feature extraction algorithm. In addition, feature extraction can also be performed by artificial neural networks such as CNN (convolutional neural network).

[0028] In some embodiments, based on a feature map of an image to be detected, a detection frame and a label corresponding to a target object in the image to be detected are generated, and the label includes location information and category information of the target object, including: inputting the feature map into a region generation network to obtain multiple candidate frames; inputting a classification network and a regression network into each of the multiple candidate frames to obtain location information and category information of each candidate frame; using a non-maximum suppression algorithm to select a target candidate frame from multiple candidate frames as a detection frame, and determining the location information and category information of the target candidate frame as a label of the detection frame.

[0029] In these implementations, the region proposal network (RPN) is used to extract multiple candidate boxes. Specifically, it performs convolution operations on the feature map through a sliding window, and then obtains multiple candidate boxes through operations such as full connection. Then, each candidate box can be input into the classification network and the regression network respectively to obtain the position information and category information of each candidate box. In practice, since the algorithm may have multiple detections on the same target, each object may have multiple candidate boxes. Therefore, it is necessary to select the most reliable candidate box from them. The process of the non-maximum suppression algorithm (NMS algorithm) is as follows: each round selects the candidate box with the highest confidence, and then pays attention to all the remaining candidate boxes that have high overlap with the selected candidate box, which will be suppressed in this round. The candidate boxes selected in this round will be retained and output, and will not appear in the next round. Then start the next round, repeat the above process, select the candidate box with the highest confidence, and suppress the high overlapping candidate box. This makes the detection of the target object more accurate, thereby improving the accuracy of image detection information.

[0030] Step 202: Determine whether the image to be detected displays a first object of a first category and a second object of a second category based on the category information.

[0031] In some embodiments, the execution subject may be based on whether the category information includes category information corresponding to the first category and the second category.

[0032] Optionally, in response to determining that the image to be detected shows the first object but does not show the second object, it can be considered that the first object is not placed on the second object, for example, it may be placed on the ground. Therefore, image detection information for characterizing that the first object is not placed on the second object in three-dimensional space can be generated.

[0033] Optionally, in response to determining that the image to be detected shows the second object but not the first object, it can be considered that the first object is not yet placed on the second object, and information indicating that the image detection is qualified can be generated as needed. At this time, subsequent steps 203 and 204 may not be performed.

[0034] Step 203 , in response to determining that the image to be detected displays the first object and the second object, based on the position information corresponding to the first object and the second object, determine the overlap rate of the detection frames corresponding to the first object and the second object respectively.

[0035] In some embodiments, in response to determining that the image to be detected displays the first object and the second object, the above-mentioned execution entity can determine the overlap rate of the detection frames corresponding to the first object and the second object respectively based on the position information corresponding to the first object and the second object respectively. Wherein, as an example, the position information can be (3, 3, 10, 10,). Wherein, "3, 3" represents the position coordinates of the detection frame, and "10, 10" represents the width and height of the detection frame. The overlap rate can be the ratio of the area of ​​the overlapping area of ​​the detection frames of the two objects to the area of ​​the detection frame of the first object.

[0036] Step 204: Generate image detection information of the image to be detected based on the overlap rate, where the image detection information is used to indicate whether the first object is placed on the second object in the three-dimensional space.

[0037] In some embodiments, the execution entity may generate image detection information of the image to be detected based on the overlap rate, where the image detection information is used to indicate whether the first object is placed on the second object in the three-dimensional space.

[0038] Optionally, an overlap rate threshold may be preset. In response to determining that the overlap rate is greater than or equal to a preset threshold (eg, 50%), image detection information is generated for characterizing that the first object is placed on the second object in the three-dimensional space.

[0039] The method provided by some embodiments of the present disclosure realizes automatic detection of whether the first object (e.g., a package) is placed on the second object (e.g., a trailer), and improves detection efficiency by avoiding manual detection. In this process, detection frames of objects of different categories are generated, and the positional relationship in three-dimensional space is determined by the detection frames. Thus, the positional relationship is determined by a single image. Compared with the related art, it does not require the combination of multiple images, simplifies the determination process, and further improves the detection efficiency.

[0040] Further references Figure 3 , which shows a process 300 of another embodiment of the image detection method. The process 300 of the image detection method includes the following steps:

[0041] Step 301: Based on the feature map of the image to be detected, a detection frame and a label corresponding to the target object in the image to be detected are generated, and the label includes the location information and category information of the target object.

[0042] Step 302: Based on the category information, determine whether the image to be detected shows a first object of a first category and a second object of a second category. The second category includes a container subcategory and a tray subcategory, so as to facilitate targeted processing for these two categories.

[0043] In some embodiments, the specific implementation of steps 301-302 and the technical effects thereof can be referred to in Figure 2 The corresponding embodiments will not be described in detail here.

[0044] Step 303 : in response to determining that the image to be detected displays the first object and the second object of the container subcategory, determining an opening area of ​​the second object in the image to be detected.

[0045] In some embodiments, since the container subcategory generally has an opening, the execution subject of the image detection method can determine the opening area of ​​the second object in the image to be detected in response to determining that the image to be detected shows the first object and the second object of the container subcategory. As an example, the image to be detected can be input into the object detection network to obtain the opening area of ​​the second object.

[0046] In an optional implementation of some embodiments, the image area in the detection frame corresponding to the second object is input into a pre-trained mask network to obtain a segmented image for characterizing the opening area. In practice, since the features of the opening area are not obvious, misidentification is prone to occur. Therefore, by segmenting the image area in the detection frame corresponding to the second object, the recognition accuracy of the opening area can be improved.

[0047] In an optional implementation of some embodiments, determining the opening area of ​​the second object in the image to be detected includes: determining a side area of ​​the second object in the image to be detected; and generating the opening area based on the side area of ​​the second object.

[0048] In these implementations, since the opening area is blocked and the recognition rate of the opening area is affected, the opening area can be indirectly determined by identifying the side area of ​​the second object and according to the structure of the second object, thereby further improving the recognition accuracy of the opening area.

[0049] Step 304 : Determine the overlap ratio between the detection frame corresponding to the first object and the opening area of ​​the second object.

[0050] In some embodiments, the execution entity may determine an overlap ratio between a detection frame corresponding to the first object and an opening area of ​​the second object.

[0051] Step 305: Generate image detection information of the image to be detected based on the overlap rate, where the image detection information is used to indicate whether the first object is placed on the second object in the three-dimensional space.

[0052] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 In the process 300 of the image detection method in some corresponding embodiments, the second objects of different subtypes are specifically identified and the overlap ratios are determined, thereby improving the accuracy of the overlap ratio and further improving the accuracy of the image detection information.

[0053] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an image detection device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0054] like Figure 4As shown, the image detection device 400 of some embodiments includes: a detection frame generation unit 401, an object determination unit 402, an overlap rate determination unit 403 and an image detection information generation unit 404. The detection frame generation unit 401 is configured to generate a detection frame and a label corresponding to the target object in the image to be detected based on the feature map of the image to be detected, and the label includes the position information and category information of the target object. The object determination unit 402 is configured to determine whether the image to be detected displays a first object of a first category and a second object of a second category based on the category information. The overlap rate determination unit 403 is configured to determine the overlap rate of the detection frames corresponding to the first object and the second object respectively based on the position information corresponding to the first object and the second object respectively in response to determining that the image to be detected displays the first object and the second object. The image detection information generation unit 404 is configured to generate image detection information of the image to be detected based on the overlap rate, and the image detection information is used to characterize whether the first object is placed on the second object in three-dimensional space.

[0055] In an optional implementation of some embodiments, the second category includes a container subcategory and a pallet subcategory; and the overlap rate determination unit 403 is further configured to: in response to determining that the image to be detected displays the first object and the second object of the container subcategory, determine the opening area of ​​the second object in the image to be detected; determine the overlap rate between the detection frame corresponding to the first object and the opening area of ​​the second object.

[0056] In an optional implementation of some embodiments, the detection frame generation unit 401 is further configured to: input the feature map into the region generation network to obtain multiple candidate frames; input each of the multiple candidate frames into the classification network and the regression network respectively to obtain the position information and category information of each candidate frame; use a non-maximum suppression algorithm to select a target candidate frame from the multiple candidate frames as the detection frame, and determine the position information and category information of the target candidate frame as the label of the detection frame.

[0057] In an optional implementation of some embodiments, the overlap rate determination unit 403 is further configured to: input the image area within the detection frame corresponding to the second object into a pre-trained mask network to obtain a segmented image for representing the opening area.

[0058] In an optional implementation of some embodiments, the overlap rate determining unit 403 is further configured to: determine a side area of ​​the second object in the image to be detected; and generate an opening area based on the side area of ​​the second object.

[0059] In an optional implementation of some embodiments, the image detection information generating unit 404 is further configured to: in response to determining that the image to be detected displays the first object and does not display the second object, generate image detection information for characterizing that the first object is not placed on the second object in the three-dimensional space.

[0060] In an optional implementation of some embodiments, the image detection information generating unit 404 is further configured to: in response to determining that the overlap rate is greater than or equal to a preset threshold, generate image detection information for characterizing that the first object is placed on the second object in the three-dimensional space.

[0061] It is understood that the units described in the device 400 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 400 and the units included therein, and will not be described in detail here.

[0062] Reference below Figure 5 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of an electronic device (electronic device in FIG. 500). Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0063] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0064] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0065] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0066] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0067] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0068] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: generates a detection frame and a label corresponding to the target object in the image to be detected based on the feature map of the image to be detected, and the label includes the position information and category information of the target object; determines whether the image to be detected displays a first object of a first category and a second object of a second category based on the category information; in response to determining that the image to be detected displays the first object and the second object, determines the overlap rate of the detection frames corresponding to the first object and the second object respectively based on the position information corresponding to the first object and the second object respectively; generates image detection information of the image to be detected based on the overlap rate, and the image detection information is used to characterize whether the first object is placed on the second object in three-dimensional space.

[0069] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0070] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0071] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a detection frame generation unit, an object determination unit, an overlap rate determination unit, and an image detection information generation unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the detection frame generation unit may also be described as a "unit for generating a detection frame and a label corresponding to a target object in an image to be detected."

[0072] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0073] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. An image detection method, comprising: Based on the feature map of the image to be detected, generating a detection frame and a label corresponding to the target object in the image to be detected, wherein the label includes location information and category information of the target object; Based on the category information, determining whether the image to be detected shows a first object of a first category and a second object of a second category, wherein the first category is a package, the second category is a package placement device, and the second category includes a container subcategory; In response to determining that the image to be detected shows the second object but does not show the first object, generating information indicating that the image detection is qualified, and terminating the execution of subsequent steps; In response to determining that the image to be detected displays a first object and a second object of the container subcategory, determining a side area of ​​the second object in the image to be detected; generating an opening area based on the side area of ​​the second object, and determining an overlap rate between a detection frame corresponding to the first object and the opening area of ​​the second object; Based on the overlap rate, image detection information of the image to be detected is generated, where the image detection information is used to indicate whether the first object is placed on the second object in a three-dimensional space.

2. The method according to claim 1, wherein: The second category includes the pallet subcategory.

3. The method according to claim 1, wherein: The step of generating a detection frame and a label corresponding to a target object in the image to be detected based on the feature map of the image to be detected, wherein the label includes location information and category information of the target object, comprises: Inputting the feature map into a region generation network to obtain multiple candidate boxes; Input the classification network and regression network into each of the multiple candidate boxes to obtain the position information and category information of each candidate box; A non-maximum suppression algorithm selects a target candidate frame from the multiple candidate frames as the detection frame, and determines the position information and category information of the target candidate frame as a label of the detection frame.

4. The method according to claim 2, wherein: The determining the opening area of ​​the second object in the image to be detected includes: The image area within the detection frame corresponding to the second object is input into a pre-trained mask network to obtain a segmented image for representing the opening area.

5. The method according to claim 1, wherein: The method further comprises: In response to determining that the image to be detected displays the first object but does not display the second object, image detection information is generated for representing that the first object is not placed on the second object in the three-dimensional space.

6. The method according to claim 1, wherein: The step of generating the image detection information of the image to be detected based on the overlap rate includes: In response to determining that the overlap ratio is greater than or equal to a preset threshold, image detection information is generated for characterizing that the first object is placed on the second object in a three-dimensional space.

7. An image detection device, comprising: A detection frame generating unit is configured to generate a detection frame and a label corresponding to a target object in the image to be detected based on a feature map of the image to be detected, wherein the label includes position information and category information of the target object; an object determination unit configured to determine, based on the category information, whether the image to be detected shows a first object of a first category and a second object of a second category, wherein the first category is a package, the second category is a package placement device, and the second category includes a container subcategory; In response to determining that the image to be detected shows the second object but does not show the first object, generating information indicating that the image detection is qualified, and terminating the execution of subsequent steps; an overlap rate determining unit, configured to, in response to determining that the image to be detected displays a first object and a second object of the container subcategory, determine a side area of ​​the second object in the image to be detected; generate an opening area based on the side area of ​​the second object, and determine an overlap rate between a detection frame corresponding to the first object and the opening area of ​​the second object; The image detection information generating unit is configured to generate image detection information of the image to be detected based on the overlap rate, wherein the image detection information is used to indicate whether the first object is placed on the second object in a three-dimensional space.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images