Parcel identification method, device and equipment

Abstract

The invention relates to a parcel identification method, device and equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a scene image at a parcel delivery place; identifying a target detection object in the scene image; comparing morphological similarity between the target detection object and a background area, wherein the background area is an area where a static object at a parcel delivery place is located; and according to the morphological similarity between the target detection object and the background area, identifying whether the target detection object is a package. By adopting the method, morphological similarity comparison can be performed on the target detection object and the background area, so that whether the target detection object is a static object or a package in a package delivery place or not can be judged according to the obtained morphological similarity, the situation that the static object is mistakenly recognized as the package can be avoided, and the recognition accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of logistics technology, and in particular to a package identification method, a package identification device and a package identification equipment. Background Art

[0002] The rapid development of the Internet and e-commerce in recent years has brought more and more convenience to people's lives, and has also driven the rapid development of logistics and related industries and products. For example, a package detection system has appeared on the market. The system is equipped with a camera with package detection function. By installing the camera at the user's door, it can detect the express parcel delivered outside the user's door, and send a notification message to the user when the express parcel is detected, thereby reducing the workload of the deliveryman.

[0003] However, when the environment at the user's door is relatively complex, for example, when there are static objects such as carpets, floor tiles, potted plants, decorations, etc. at the user's door, the package detection system may mistakenly identify these static objects as packages. This misjudgment not only reduces the reliability of the package detection system, but also brings unnecessary interference to the user. Summary of the invention

[0004] Based on this, it is necessary to provide a package identification method, device and equipment that can improve the accuracy of package identification in response to the above technical problems.

[0005] In a first aspect, the present application provides a package identification method. The method comprises:

[0006] Obtain scene images of the package delivery location;

[0007] Identifying a target detection object in the scene image;

[0008] Comparing the morphological similarity between the target detection object and a background area, wherein the background area is an area where static objects at the package delivery location are located;

[0009] According to the morphological similarity between the target detection object and the background area, it is identified whether the target detection object is a package.

[0010] In one embodiment, the method of obtaining the background area includes:

[0011] Acquire a static image of a scene of the package delivery without delivering the package;

[0012] The area where the static object in the static image of the scene is located is marked to obtain the background area.

[0013] In one embodiment, marking the area where the static object is located in the static image of the scene to obtain the background area includes:

[0014] Acquire background area designation information, the background area designation information comprising: relevant information of static objects in the scene static image determined based on the scene static image;

[0015] The scene static image is marked according to the background area designation information, and the marked area is used as the background area.

[0016] In one embodiment, comparing the morphological similarity between the target detection object and the background area includes:

[0017] Compare the intersection-over-union ratio between the target detection object and the background area.

[0018] In one embodiment, comparing the intersection-over-union ratio between the target detection object and the background area includes:

[0019] Extracting feature points of the target detection object and the background area;

[0020] The feature points of the target detection object and the feature points of the background area are compared, and the intersection-and-union ratio of the target detection object and the background area is determined according to the feature point comparison result.

[0021] In one of the embodiments, the background area includes: a background frame obtained by marking an area where a static object in a static image of the scene is located as a priori frame, and the static image of the scene is a scene image of the package delivery in a state where the package is not delivered.

[0022] In one embodiment, after detecting whether the target object is a package, the method further includes:

[0023] If the target object is not a package, sending a query message whether the target object is a package;

[0024] When a reply message indicating that the target detection object is not a package is received based on the query message, the target detection object is marked as a background area.

[0025] In one embodiment, after identifying whether the target object is a package, the method further includes:

[0026] If the target detection object is a package, a package delivery prompt message is sent to the connected user terminal.

[0027] In one embodiment, obtaining a scene image of a parcel delivery location includes:

[0028] When a human body signal is detected, a scene image of the parcel delivery location is acquired; the human body signal is emitted by the human body detection module when a person is detected.

[0029] In a second aspect, the present application also provides a package identification device. The device comprises:

[0030] An image acquisition module, used to acquire scene images of a parcel delivery location;

[0031] A detection object recognition module, used to recognize the target detection object in the scene image;

[0032] A similarity comparison module, used to compare the morphological similarity between the target detection object and the background area, wherein the background area is the area where the static objects at the package delivery location are located;

[0033] The package identification module is used to identify whether the target detection object is a package according to the morphological similarity between the target detection object and the background area.

[0034] In a third aspect, the present application further provides a package identification device. The device comprises: a control module and a camera device, wherein the camera device is connected to the control module;

[0035] The camera device is used to collect scene images at the parcel delivery location;

[0036] The control module is used to obtain a scene image of a package delivery location from the camera device; identify a target object in the scene image; compare the morphological similarity between the target object and a background area, where the background area is an area where static objects at the package delivery location are located; and identify whether the target object is a package based on the morphological similarity between the target object and the background area.

[0037] In one of the embodiments, the device further comprises a human body detection module, and the human body detection module is connected to the control module;

[0038] The human body detection module is used to send a human body signal when a person is detected at the package delivery location;

[0039] The control module is further used to control the camera device to collect the scene image of the package delivery location when receiving the human body signal, and obtain the scene image of the package delivery location from the camera device.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0041] Obtain scene images of the package delivery location;

[0042] Identifying a target detection object in the scene image;

[0043] Comparing the morphological similarity between the target detection object and a background area, wherein the background area is an area where static objects at the package delivery location are located;

[0044] According to the morphological similarity between the target detection object and the background area, it is identified whether the target detection object is a package.

[0045] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0046] Obtain scene images of the package delivery location;

[0047] Identifying a target detection object in the scene image;

[0048] Comparing the morphological similarity between the target detection object and a background area, wherein the background area is an area where static objects at the package delivery location are located;

[0049] According to the morphological similarity between the target detection object and the background area, it is identified whether the target detection object is a package.

[0050] The above-mentioned package identification method, device, equipment, computer-readable storage medium and computer program product obtain a scene image of a package delivery location; identify a target detection object in the scene image; compare the morphological similarity between the target detection object and the background area, where the background area is the area where static objects at the package delivery location are located; and identify whether the target detection object is a package based on the morphological similarity between the target detection object and the background area. Thus, by comparing the morphological similarity between the target detection object and the background area, it is possible to determine whether the target detection object is a static object or a package at the package delivery location based on the obtained morphological similarity, thereby avoiding the situation where a static object is mistakenly identified as a package, thereby improving the accuracy of identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a module of a package identification device in one embodiment;

[0052] Figure 2 It is a schematic diagram of modules of a package identification device in another embodiment;

[0053] Figure 3 is a schematic flow chart of a package identification method in one embodiment;

[0054] Figure 4 is a flow chart of a package identification method in another embodiment;

[0055] Figure 5 It is a flowchart of a method for obtaining a background area in a package identification method in one embodiment;

[0056] Figure 6 A schematic diagram of a process of marking an area where a static object is located in a static image of a scene to obtain a background area in an embodiment;

[0057] Figure 7 is a flowchart of a package identification method in yet another embodiment;

[0058] Figure 8 A schematic diagram of a flow chart of a package identification method in yet another embodiment;

[0059] Fig. 9 A schematic diagram of a process of a package identification method in another embodiment;

[0060] Fig.10 is a structural block diagram of a package identification device in one embodiment;

[0061] Fig.11 FIG. 4 is a structural block diagram of a package identification device in another embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] At present, in order to solve the safety problem of express parcels, most camera devices on the market are equipped with AI (Artificial Intelligence) parcel detection function. The parcel detection system equipped with a camera with parcel detection function monitors the user's door environment in real time to identify the express parcels in the door environment, and can automatically send a notification to the user after identifying the express parcel. However, based on the actual scene of the user's door and the particularity of the parcel delivery process, the application of this technology still has the following two pain points:

[0064] 1. For users whose door environment is relatively complex, static objects placed at the door are often mistakenly identified as packages. Specifically, when the door environment is relatively complex, such as when there are static objects such as carpets, floor tiles, potted plants, and decorations, the package detection system may mistakenly identify these static objects as packages. This is because static objects and packages are similar in shape and size, and are easily confused, resulting in misjudgment. For users whose door environment is relatively complex, this kind of misjudgment may occur frequently, reducing the accuracy and reliability of package detection, and also causing unnecessary interference and distress to users.

[0065] 2. When the morphological similarity of a static object is highly overlapped with the target object, there will be a problem of missed detection. Specifically, when the morphological similarity of a static object is highly overlapped with the target object, for example, a rectangular package is placed on a rectangular carpet of similar size. Since the shape and size of the rectangular package are very similar to the rectangular carpet, the system will directly default the package to a carpet based on the shape and size of the package, resulting in a problem of missed detection. This missed detection situation will cause unnecessary trouble to the user, such as delaying the pickup time of the package, or the user will not be able to pick up the package in time, and the package will be lost or stolen.

[0066] Based on the above problems, an embodiment of the present application provides a package identification method, which first obtains a scene image of a package delivery location; identifies a target detection object in the scene image; compares the morphological similarity between the target detection object and the background area, where the background area is the area where static objects at the package delivery location are located; and identifies whether the target detection object is a package based on the morphological similarity between the target detection object and the background area. Thus, by comparing the morphological similarity between the target detection object and the background area, it is possible to determine whether the target detection object is a static object or a package at the package delivery location based on the obtained morphological similarity, thereby avoiding the situation where a static object is mistakenly identified as a package, thereby improving the accuracy of identification.

[0067] The package identification method provided in the embodiment of the present application can be applied to a package identification device. Figure 1 As shown, the package identification device includes a control module 110 and a camera device 120, and the camera device 120 is connected to the control module 110. The camera device 120 is used to collect scene images at the package delivery site. The control module 110 is used to obtain the scene image of the package delivery site from the camera device 120; identify the target detection object in the scene image; compare the morphological similarity between the target detection object and the background area, and the background area is the area where the static objects at the package delivery site are located; according to the morphological similarity between the target detection object and the background area, identify whether the target detection object is a package. Among them, the control module 110 can also communicate with the user's terminal, and the terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0068] In actual implementation, the structure of the package identification device can also be adaptively modulated according to the implementation needs. For example, as shown in Figure 2, the package identification device can also include a human body detection module 130, which is connected to the control module 110. The human body detection module 130 is used to send a human body signal to the control module 110 when a person appears at the package delivery location; the control module 110 is also used to control the camera device 120 to collect the scene image of the package delivery location when receiving the human body signal, and obtain the scene image of the package delivery location from the camera device 120.

[0069] In one embodiment, Figure 3 As shown, a package identification method is provided, which is applied to Figure 1 The control module in is taken as an example to illustrate, including the following steps 100-400.

[0070] Step 100, obtaining a scene image of a parcel delivery location.

[0071] Among them, parcels include objects delivered to users by delivery methods such as express delivery and takeout. The parcel delivery point is the delivery location of the parcel, that is, the place where the user picks up the parcel. The specific location of the parcel delivery point can be determined according to the delivery needs of different users, such as the user's door, express delivery station, etc. When the parcel delivery point is the user's door, it can specifically be the area outside the user's entrance door. The deliveryman only needs to deliver the parcel to the user's door. This not only improves the delivery efficiency of the deliveryman, but also makes it convenient for users to pick up at any time. For the sake of convenience of explanation, the following description is based on the example of the parcel delivery point being the user's door.

[0072] In this embodiment, the camera device is a device capable of capturing video images. For example, the camera device may be a security camera or a doorbell or door lock with a camera. The camera device is installed at the user's door and can capture scene images of the user's door area after installation. The camera device can continuously capture video images of the user's door, or it can start capturing video images when it is awakened. After capturing the scene image, it sends the scene image to the control module in real time.

[0073] Step 200: Identify the target object in the scene image.

[0074] The control module may be preset with an image processing algorithm and a target detection algorithm. After acquiring the scene image, each frame of the video image is first extracted based on the image processing algorithm, and then the target detection object in each frame of the image is detected and identified based on the target detection algorithm. The "target detection object" in the present embodiment is set as a package. In the process of presetting the target detection algorithm, a target detection model can also be obtained based on the target detection algorithm and a large amount of package-related data training. The target detection model can identify whether there is a target detection object (package) in each frame of the input image, and output the corresponding detection result. After the target detection object (package) is identified, its output result can be a detection frame, and the target detection object is marked with a detection frame.

[0075] Step 300, comparing the morphological similarity between the target object and the background area.

[0076] The background area is the area where static objects are located at the parcel delivery location. Static objects refer to objects that do not move in the user's door environment, such as carpets, floor tiles, potted plants, decorative objects, etc. The background area is used to mark the area where each static object is located. It is pre-stored in the control module. During the pre-storage process, the size, proportion, and quantity of the background area parameters need to be adjusted according to the type and quantity of static objects in the user's door environment.

[0077] Morphological similarity refers to the degree of similarity between two objects in terms of shape, size, color, etc. In one embodiment, comparing the morphological similarity between the target object and the background area includes: comparing the intersection-and-union ratio between the target object and the background area.

[0078] Intersection-over-Union (IoU) is a concept in target detection, which is the overlap ratio between the generated prediction bound and the original labeled bound, that is, the ratio of their intersection to their union. Specifically in this embodiment, the IoU obtained by comparing the target detection object and the background area is the ratio between the intersection area of ​​the target detection object and the background area and their union area, and this ratio is used as the IoU.

[0079] Step 400, identifying whether the target object is a package based on the morphological similarity between the target object and the background area.

[0080] It can be understood that the greater the morphological similarity between the target detection object and the background area, the higher the overlap between the target detection object and the background area, and the more similar the target detection object is to the static object in the background area, that is, the greater the possibility that the target detection object is a static object. Therefore, in order to reduce the false detection rate, the control module can also set a preset recognition threshold according to actual needs, and step 400 can include: identifying whether the target detection object is a package according to the morphological similarity and the preset recognition threshold.

[0081] Specifically, in one embodiment, the target object is identified as a package based on the morphological similarity and the preset recognition threshold, including: when the morphological similarity between the target object and the background area reaches the preset recognition threshold, the target object is determined to be not a package; when the morphological similarity between the target object and the background area is lower than the preset recognition threshold, the target object is determined to be a package. Thus, by comparing the morphological similarity between the target object and the background object, when the morphological similarity between the two reaches the preset recognition threshold, it can be determined that the target object detected in the scene image is actually a static object at the package delivery location; only when the morphological similarity between the two is lower than the preset recognition threshold, that is, the target object is not a static object, is it identified as a package. This can avoid the situation where static objects are mistakenly identified as packages, thereby improving the accuracy of identification.

[0082] In another embodiment, identifying whether the target object is a package based on the morphological similarity and a preset recognition threshold includes: when the morphological similarity between the target object and the background area is lower than the preset recognition threshold, detecting whether the target object is a package.

[0083] It can be understood that the control module is also preset with a package recognition algorithm. In the preset process, a package recognition model is obtained by training based on the package recognition algorithm and package-related data. The package recognition model can accurately detect whether the target detection object is a package. In this embodiment, when the morphological similarity between the two is lower than the preset recognition threshold, it is determined that the target detection object is not a static object, but a newly added object in the doorway area that may be a package. At this time, the next step of detection processing can be combined with the package recognition model to accurately detect whether the detection target detection object is a package.

[0084] Therefore, when the morphological similarity between the target object and the background area is lower than the preset recognition threshold, it is determined that the target object is not a static object, but a newly added object at the parcel delivery location. At this time, the target object will be detected to see if it is a parcel, which can avoid missed detection. Therefore, the parcel recognition method can effectively reduce the false detection rate and missed detection rate, improve the detection reliability, avoid unnecessary troubles for users, and bring better user experience.

[0085] The above-mentioned package recognition method compares the morphological similarity between the target detection object and the background area, and can determine whether the target detection object is a static object or package in the package delivery area based on the obtained morphological similarity, thereby avoiding the situation where static objects are mistakenly identified as packages, thereby improving the accuracy of recognition.

[0086] In one embodiment, Figure 4 As shown, if the recognition result is that the target object is a package, step 400 also includes step 500: sending package delivery prompt information to the connected user terminal.

[0087] Specifically, if the recognition result is that the target object is a package, the control module sends a package delivery prompt message to the connected user terminal to prompt the user, so that the user can promptly learn about the package delivery information and collect the package in time. If the recognition result is that the target object is not a package, the control module does not send a prompt message to avoid unnecessary disturbance to the user.

[0088] Exemplarily, an APP (Application) may be installed on the user's terminal. After the control module sends the package delivery reminder information to the user's terminal, the corresponding package information is displayed through the APP, so that the user can timely understand the delivery status of the package.

[0089] In one embodiment, Figure 5 As shown, in the package identification method, the method for obtaining the background area includes steps 600 and 700.

[0090] Step 600, obtaining a static image of a scene in which a package is not delivered at a package delivery location.

[0091] Step 700: Mark the area where the static object in the static image of the scene is located to obtain the background area.

[0092] The scene static image can be uploaded by the user to the APP on the terminal, and then uploaded to the control module through the APP. The scene static image can also be set or updated by the user based on the video stream of the camera. In actual implementation, the user can enter the video stream of the camera through the APP interface, and then save the interface information on the video stream interface as the scene static image.

[0093] It is understandable that during holidays such as the Spring Festival and Christmas, some decorative static objects may be added to the user's door area. In order to improve the accuracy of detection, the user can update the scene static image in time when the door environment changes.

[0094] There are many ways to mark the area where the static object is located in the static image of the scene. In one embodiment, Figure 6 As shown, step 700 includes step 710 and step 720 .

[0095] Step 710, obtaining background area designation information.

[0096] Step 720: Mark the scene static image according to the background area designation information, and use the marked area as the background area.

[0097] The background area designation information includes: information related to static objects in the scene static image determined based on the scene static image. The information related to the static objects can be set in accordance with actual conditions, for example, including coordinate information, size information, type information, etc. of the static objects.

[0098] In one embodiment, the background area designation information is sent by the user through the APP, specifically, Figure 7 As shown, the user can manually select the background frame of the area to be set as the background on the APP side based on the static image of the scene, and the coordinate information of the selected background frame is uploaded by the APP side to the control module. After the target detection model pre-trained in the control module performs target detection based on the scene image, the output detection frame representing the target detection object is compared with the background frame manually selected by the user. When the intersection and union ratio of the two is greater than or equal to the preset recognition threshold (the figure takes the preset recognition threshold of 0.95 as an example), the target detection object is considered to be a static object in the background area. At this time, the detection result is not pushed to the user. When the intersection and union ratio of the two is lower than the preset recognition threshold, it is further identified whether the target detection object in the detection frame is a package. If it is a package, the notification information of the package delivery is pushed to the user.

[0099] When the user selects the background frame to be used as the background area, the user needs to select the area where the static objects that are easily confused as packages are located, so that the accuracy of the background area marking can be ensured by the user manually marking the background frame. Especially for the situation where the environment at the user's door is more complicated, accurate marking of the background area can effectively improve the reliability of detection.

[0100] In another embodiment, the background area can also be automatically identified by the APP end, and the designated information is the relevant information of the static objects in the static image of the scene determined by the APP end based on the static image of the scene. Specifically, after receiving the static image of the scene, the APP end identifies the static objects in the static image of the scene; compares the morphological similarity between the static object and the package, and when the morphological similarity between the static object and the package is higher than the preset background threshold, the area where the static object is located is marked as the background frame of the background area. Among them, the preset background threshold needs to be set according to the actual situation, for example, 0.95. Furthermore, the APP end will send the coordinate information of the background frame to the control module, so that the control module determines the background area according to the coordinate information.

[0101] In other embodiments, the method for marking the area where the static object in the static image of the scene is located can also be that after receiving the static image of the scene, the control module identifies the static object in the static image of the scene; compares the morphological similarity between the static object and the package, and when the morphological similarity between the static object and the package is higher than a preset background threshold, marks the area where the static object is located, and the marked area serves as the background area.

[0102] By acquiring a static image of the scene at the parcel delivery site when no parcel has been delivered, and determining the background area based on the static image of the scene, similar objects in the background area that are easily confused with the parcel can be marked, thereby reducing the false detection rate and missed detection rate when detecting the target object and improving the reliability of detection.

[0103] In one embodiment, comparing the intersection-and-union ratio between the target object and the background area includes: extracting feature points of the target object and the background area; comparing the feature points of the target object with the feature points of the background area, and determining the intersection-and-union ratio between the target object and the background area based on the feature point comparison result.

[0104] Among them, the feature point may refer to a point with significant changes in the target detection object and the background area. In one embodiment, when extracting the feature points of the target detection object and the background area, the feature points of the shape features of the two areas can be extracted respectively, and then the feature points of the two areas are compared for overlap, and the feature point comparison result is used as the intersection and union comparison result of the target detection object and the background area. The shape features may include features such as the outline, the aspect ratio of the bounding box, and the geometric shape, which can be used to distinguish the shape differences between the target detection object and the background area.

[0105] In this embodiment, by comparing multiple feature points of the target detection object and the background area, the two can be analyzed and detected at a deeper level, and the detection results can be quantified by the intersection-union ratio. The intersection-union ratio obtained in this way is more reliable, thereby making the package recognition result more accurate.

[0106] Further, when extracting the feature points of the target detection object and the background area, at least one of the color features, structural features and depth features of the two can also be extracted respectively. According to the extracted color feature points, the color distribution of the target detection object and the background area can be compared, and the color feature overlap of the target detection object and the background area can be compared using statistical information such as color histogram, color moment, color channel, etc. According to the extracted structural feature pairs, the spatial distribution, relative position, etc. between the target detection object and the background area can be compared to determine the overlap of the structural information between the two areas. When comparing the depth features, a deep learning model can be used to extract feature representations from the two areas, and by comparing these feature representations, the differences in feature points such as shape structure between the two areas can be judged. Thus, by comparing the multiple feature points of the target detection object and the background area, a more comprehensive analysis and detection of the two areas can be achieved, so that the intersection and union results are more accurate.

[0107] In one embodiment, the background area includes: a background frame obtained by marking the area where the static object in the static image of the scene is located as a priori frame.

[0108] Anchor box is a concept in object detection, which can be regarded as a reference box used to indicate the possible locations of objects of different sizes and proportions in an image. In actual implementation, the anchor box includes but is not limited to a rectangular box, a hexagonal box, etc., which can be set according to the shape of the static object.

[0109] In this embodiment, multiple prior frames of different sizes and proportions are predefined according to static objects that are easily confused as packages. After target detection is performed based on the scene image to obtain the detection frame, the intersection-and-union ratio of the detection frame and the prior frame is compared. When the intersection-and-union ratio is greater than the preset recognition threshold, that is, when the background frame and the prediction frame have a high degree of overlap, the detection frame is more likely to be a static object. At this time, the prediction result of the target detection model will not execute the next step of determining whether there is a package in the prediction frame, and vice versa.

[0110] Therefore, by marking static objects or high-frequency false detection areas as prior frames, we can avoid the problem of false detection of static objects close to the package and solve the problem of missed detection of targets superimposed with the background, thereby improving the accuracy of package recognition and further improving the utilization rate of package recognition equipment.

[0111] In one embodiment, step 400 includes detecting whether the target object is a package when the morphological similarity between the target object and the background area is lower than a preset recognition threshold, such as Figure 8 As shown, after step 400, if the target object is not a package, the package identification method further includes steps 800 and 810.

[0112] Step 800: Send inquiry information to determine whether the target object is a package.

[0113] Step 810: When receiving a reply message indicating that the target detection object is not a package based on the query message, mark the target detection object as a background area.

[0114] In actual implementation, when the control module determines that the target detection object is not a package during further detection, it is believed that the target detection object may only be a similar object to the package. In order to avoid false detection, the control module can send an inquiry message to the user APP end at this time, and the user manually determines whether the target detection object in the target detection object is a package. If the user's judgment result is no, it means that the object in the target detection object is indeed a similar object to the package. It is possible that the user missed the mark when manually marking. In order to avoid the misidentification of the similar object in the future, the target detection object can be supplemented with a background area. If the user's judgment result is yes, it means that the target detection object is a package, so that missed detection can be avoided by manual confirmation.

[0115] Further, please refer to Fig. 9 As shown, in the recognition process of the package recognition method, each frame image of the scene image is extracted, and the target detection model recognizes the target detection object in each frame image to obtain multiple detection frames (illustrated as detection frame a-detection frame n), and then compares the morphological similarity between the target detection object (each detection frame) and the background area (illustrated as the background frame) in each frame image. When the morphological similarity between the target detection object and the background area is lower than the preset recognition threshold, it is detected whether the target detection object in each frame image is a package. If multiple frames ( Fig. 9 Taking 4 frames as an example), the target object in the image is in the same area where the package is delivered, and the morphological similarity between the target object and the background area is not less than the preset recognition threshold (such as 0.95). After re-detection, it is determined that the target object is not a package. At this time, the control module will send an inquiry message to the APP to determine whether the target object is a package, and the user will determine whether the target object that is frequently triggered is a package. If the user determines that this is not a package, the detection frame will be marked as a background area to avoid misidentification next time, which can also reduce unnecessary trouble to the user.

[0116] In one embodiment, step 200 includes: when a human body signal is detected, acquiring a scene image of a package delivery location.

[0117] The human signal is emitted by the human detection module when a person is detected. The human detection module may specifically include a PIR (Passive Infrared Sensor) sensor. The PIR sensor may be installed in the user's door area to detect whether there is a person in the area.

[0118] In actual implementation, when the PIR sensor does not detect a human signal, the camera device can be in standby or sleep mode. When the PIR sensor detects a person in the doorway area, since the person in the doorway area may be a deliveryman delivering a package, in order to avoid missed detection, the PIR sensor will send the corresponding human signal to the control module, and the control module will wake up the camera device to start recording the real-time scene image of the doorway area and send it to the control module. After the control module obtains the recognition result based on the scene image, it can control the camera device to re-enter the standby or sleep mode, thereby saving energy consumption and storage space, and effectively reducing unnecessary calculations.

[0119] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a package identification device for implementing the package identification method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more package identification device embodiments provided below can refer to the limitations of the package identification method above, and will not be repeated here.

[0121] In one embodiment, Fig.10 As shown, a package identification device is provided, including: an image acquisition module 910, a detection object identification module 920 and a package identification module 940, wherein:

[0122] The image acquisition module 910 is used to acquire scene images of the parcel delivery location.

[0123] The detection object recognition module 920 is used to recognize the target detection object in the scene image.

[0124] A similarity comparison module 930 is used to compare the morphological similarity between the target object and the background area, where the background area is the area where static objects are located at the package delivery location;

[0125] The package identification module 940 is used to identify whether the target detection object is a package according to the morphological similarity between the target detection object and the background area.

[0126] In one embodiment, Fig.11 As shown, the package identification device further includes a marking module 950, which is used to obtain a static image of a scene in which a package is not delivered; and mark the area where the static object is located in the static image of the scene to obtain a background area.

[0127] In one embodiment, the marking module 950 is also used to obtain background area designation information, which includes: relevant information of static objects in the scene static image determined based on the scene static image; marking in the scene static image according to the background area designation information, and using the marked area as the background area.

[0128] In one embodiment, the similarity comparison module 930 is further used to compare the intersection-over-union ratio between the target detection object and the background area.

[0129] In one embodiment, the similarity comparison module 930 is also used to extract feature points of the target object and the background area; compare the feature points of the target object and the feature points of the background area, and determine the intersection and union ratio of the target object and the background area based on the feature point comparison results.

[0130] In one embodiment, the package identification module 940 is further configured to detect whether the target object is a package when the morphological similarity between the target object and the background area is lower than a preset identification threshold.

[0131] In one embodiment, if the detected target object is not a package, the package identification module 940 is further used to send an inquiry message whether the target object is a package; when a reply message indicating that the target object is not a package is received based on the inquiry message, the target object is marked as a background area.

[0132] In one embodiment, if the identified target object is a package, the package identification module 940 is further configured to send package delivery prompt information to a connected user terminal.

[0133] In one embodiment, the image acquisition module 910 is further used to acquire a scene image of the package delivery location when a human signal is detected; the human signal is emitted by the human detection module when a person is detected.

[0134] Each module in the above-mentioned package identification device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0135] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0136] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0138] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0139] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A package identification method, characterized in that: The method comprises: Obtain scene images of the package delivery location; Identifying a target detection object in the scene image; Comparing the morphological similarity between the target detection object and a background area, wherein the background area is an area where static objects at the package delivery location are located; According to the morphological similarity between the target detection object and the background area, it is identified whether the target detection object is a package.

2. The method according to claim 1, characterized in that The method for obtaining the background area includes: Acquire a static image of a scene of the package delivery without delivering the package; The area where the static object in the static image of the scene is located is marked to obtain the background area.

3. The method according to claim 2, characterized in that The step of marking the area where the static object in the static image of the scene is located to obtain the background area includes: Acquire background area designation information, the background area designation information comprising: relevant information of static objects in the scene static image determined based on the scene static image; The scene static image is marked according to the background area designation information, and the marked area is used as the background area.

4. The method according to claim 1, characterized in that: The comparing the morphological similarity between the target detection object and the background area includes: Compare the intersection-over-union ratio between the target detection object and the background area.

5. The method according to claim 4, characterized in that The comparing the intersection-and-union ratio between the target detection object and the background area includes: Extracting feature points of the target detection object and the background area; The feature points of the target detection object and the feature points of the background area are compared, and the intersection-and-union ratio of the target detection object and the background area is determined according to the feature point comparison result.

6. The method according to claim 1, characterized in that The background area includes: a background frame obtained by marking the area where the static object in the scene static image is located as a priori frame, and the scene static image is a scene image of the package delivery in a state where the package is not delivered.

7. The method according to claim 1, characterized in that The step of identifying whether the target detection object is a package according to the morphological similarity between the target detection object and the background area includes: When the morphological similarity between the target detection object and the background area is lower than a preset recognition threshold, it is detected whether the target detection object is a package.

8. The method according to claim 7, characterized in that After detecting whether the target object is a package, the method further includes: If the target object is not a package, sending a query message whether the target object is a package; When a reply message indicating that the target detection object is not a package is received based on the query message, the target detection object is marked as a background area.

9. The method according to claim 1, characterized in that: After identifying whether the target object is a package, the method further includes: If the target detection object is a package, a package delivery prompt message is sent to the connected user terminal.

10. The method according to any one of claims 1 to 9, characterized in that: The step of obtaining a scene image of a parcel delivery location includes: When a human body signal is detected, a scene image of the parcel delivery location is acquired; the human body signal is emitted by the human body detection module when a person is detected.

11. A package identification device, characterized in that: The device comprises: An image acquisition module, used to acquire scene images of a parcel delivery location; A detection object recognition module, used to recognize the target detection object in the scene image; A similarity comparison module, used to compare the morphological similarity between the target detection object and the background area, wherein the background area is the area where the static objects at the package delivery location are located; The package identification module is used to identify whether the target detection object is a package according to the morphological similarity between the target detection object and the background area.

12. A package identification device, characterized in that: The device comprises: a control module and a camera device, wherein the camera device is connected to the control module; The camera device is used to collect scene images at the parcel delivery location; The control module is used to obtain a scene image of a package delivery location from the camera device; identify a target object in the scene image; compare the morphological similarity between the target object and a background area, where the background area is an area where static objects at the package delivery location are located; and identify whether the target object is a package based on the morphological similarity between the target object and the background area.

13. The device according to claim 12, characterized in that The device also includes a human body detection module, which is connected to the control module; The human body detection module is used to send a human body signal when a person is detected at the package delivery location; The control module is further used to control the camera device to collect the scene image of the package delivery location when receiving the human body signal, and obtain the scene image of the package delivery location from the camera device.

Images