Package detection method and device

Abstract

The present invention discloses a package detection method and device. Its features include: obtaining a video to be detected, extracting an image to be detected in the video to be detected; performing occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image; inputting the target detection image into a pre-trained package detection model to obtain a package detection image. This achieves the exclusion of images in non-detection areas, highlights the features of the target detection area of ​​the image, reduces the amount of image data, improves the accuracy of image detection, reduces the probability of package loss and delay, and improves user satisfaction.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a package detection method and device. Background Art

[0002] In today's society, express delivery is one of the basic services in daily life, and people have high requirements for the speed and accuracy of express delivery. In the process of ensuring high-quality express delivery services, parcel sorting is an important part. Efficient and accurate parcel sorting is conducive to improving the service quality of express delivery.

[0003] In the process of parcel sorting, the baffles in the conveyor belt are generally used to sort the parcels. In the related art, the parcel sorting video is manually inspected to determine when the parcel is spilled, or the parcel recognition algorithm is used to inspect each parcel separately to determine whether the parcel is spilled. Both methods have low detection efficiency and are prone to spilling parcels or detecting parcel errors. Summary of the invention

[0004] The present invention provides a package detection method and device to achieve high-efficiency and high-accuracy detection of packages.

[0005] According to one aspect of the present invention, a package detection method is provided, comprising:

[0006] Acquire a video to be detected, and extract an image to be detected from the video to be detected;

[0007] Performing occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image;

[0008] The target detection image is input into a pre-trained package detection model to obtain a package detection image.

[0009] According to another aspect of the present invention, there is provided a package detection device, comprising:

[0010] An image extraction module is used to obtain a video to be detected and extract an image to be detected from the video to be detected;

[0011] An image processing module is used to perform occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image;

[0012] The package detection module is used to input the target detection image into a pre-trained package detection model to obtain a package detection image.

[0013] The technical solution of the embodiment of the present invention obtains a video to be detected, extracts an image to be detected in the video to be detected, performs occlusion processing on the area except the area to be detected in the image to be detected, excludes the image of the non-detection area from the area that does not need to be detected, and obtains a target detection image, thereby highlighting the features of the target area that needs to be detected and reducing the amount of image data. The target detection image is input into a pre-trained package detection model to obtain a package detection image. The technical problems in the prior art that the detection efficiency is too slow and it is easy to scatter packages or detect packages incorrectly are solved, thereby improving the accuracy of image detection, reducing the probability of package loss and delay, and improving user satisfaction.

[0014] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 is a flow chart of a package detection method provided by Embodiment 1 of the present invention;

[0017] Figure 2 is a flow chart of another package detection method provided by Embodiment 2 of the present invention;

[0018] Figure 3 is a flow chart of another package detection method provided by Embodiment 3 of the present invention;

[0019] Figure 4 is a structural diagram of a package detection device provided by Embodiment 4 of the present invention;

[0020] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing the package detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] Embodiment 1

[0023] Figure 1 A flowchart of a parcel detection method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of parcel identification detection in parcel sorting. The method can be performed by a parcel detection device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0024] S110: Acquire a video to be detected, and extract an image to be detected from the video to be detected.

[0025] The video to be detected may be a video of the parcel sorting process shot by a shooting device during the parcel sorting process.

[0026] Specifically, during the parcel sorting process, multiple shooting devices are used to shoot all videos of the entire parcel sorting process in all directions, and then a target shooting device is selected from the multiple shooting devices to shoot the area to be detected, and then the video of the area to be detected shot by the target shooting device is used as the video to be detected. An image is extracted from the video to be detected as the image to be detected. Optionally, a picture is randomly captured as the image to be detected when the video to be detected is played.

[0027] Optionally, in another embodiment of the present invention, extracting the image to be detected from the video to be detected includes at least one of the following operations:

[0028] Optionally, the image to be detected is extracted from the video to be detected according to a preset image extraction frequency; wherein the preset image extraction frequency may be an arbitrarily set extraction frequency, for example, 2 images may be extracted every 1 second, 10 images may be extracted within 1 minute, and 1000 images may be extracted within 1 hour, etc.; the image to be detected is randomly extracted within a preset unit time according to the preset image extraction frequency, thereby completing the number of images to be detected that need to be extracted within the unit time.

[0029] Optionally, a preset number of images to be detected are randomly extracted from the video to be detected, wherein the preset number may be a preset number of images, and the user may input a preset number of images to be extracted before extracting the images to be detected, so that a preset number of images to be detected are randomly extracted from the video to be detected.

[0030] Optionally, the image to be detected is extracted from the video to be detected according to a preset extraction time interval; wherein the preset extraction time interval may be a preset time interval for extracting images, such as 5ms or 10ms. The user may input the preset extraction time interval before extracting the image to be detected, and then extract an image to be detected from the video to be detected every preset extraction time interval.

[0031] Optionally, each frame of the image in the video to be detected is extracted as an image to be detected.

[0032] S120, performing occlusion processing on the area other than the to-be-detected area in the to-be-detected image to obtain a target detection image.

[0033] Considering that the conveyor belt is needed to transport the parcels during the parcel sorting process, it is a normal parcel sorting process. Parcel spillage events are often used to detect parcels that fall outside the conveyor belt. In other words, parcels appear in the area to be detected. In order to prevent the parcels on the conveyor belt from affecting the detection results, the area other than the area to be detected in the image to be detected can be blocked, laying the foundation for accurately detecting parcel spillage events.

[0034] Specifically, when the image to be detected is obtained, the position information of the area to be detected in the image to be detected is determined, and then the area to be detected is demarcated in the image to be detected, and then the area other than the area to be detected in the image to be detected is blocked, and then the target detection image is obtained. Among them, the blocking of the area other than the area to be detected can be blurring the image to block the area other than the area to be detected.

[0035] Optionally, in another embodiment of the present invention, the occlusion processing of the area other than the detection area in the image to be detected includes: based on pre-set coordinates corresponding to the area to be detected in the image to be detected, performing occlusion processing on the area other than the detection area in the image to be detected.

[0036] Specifically, based on the pre-set coordinates corresponding to the area to be detected in the image to be detected, the position of the area to be detected in the image to be detected is determined, and then the area in the image to be detected except the area to be detected is blocked. Optionally, after determining the corresponding coordinates of the area to be detected in the image to be detected, the coordinates of each pixel in the area of ​​the image to be detected except the area to be detected are determined according to the position information in the image to be detected and the corresponding coordinates of the area of ​​the image to be detected except the area to be detected, and the image is replaced by the coordinates of each pixel in the area of ​​the image to be detected except the area to be detected, and the area except the area to be detected is blocked.

[0037] In order to realize fast masking of the image, a preset masking frame may be used to mask the area other than the area to be detected in the image to be detected. For example, the masking frame may be a rectangular frame with a preset filling color.

[0038] Optionally, the masking process for the area other than the area to be detected in the image to be detected may also be: receiving a mask frame adding operation for the image to be detected, displaying a preset mask frame in the image to be detected; and receiving a mask adjustment operation input for the preset mask frame to mask the area other than the area to be detected in the image to be detected. The mask adjustment operation at least includes an operation of adjusting the size of the preset mask frame and / or an operation of adjusting the position of the preset mask frame.

[0039] Optionally, the area other than the area to be detected in the image to be detected may be blurred to cover the area other than the area to be detected in the image to be detected.

[0040] S130: Input the target detection image into a pre-trained package detection model to obtain a package detection image.

[0041] The package detection model may be a neural network model for detecting whether there is a scattered package in the target detection image. Exemplarily, the package training model may include but is not limited to: at least one of Convolutional Neural Networks (CNN), Fully Connected Neural Networks (FCN), Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM).

[0042] Specifically, the target detection image is input into a pre-trained package detection model, and the package detection model outputs a package detection image.

[0043] The technical solution of the embodiment of the present invention obtains a video to be detected, extracts an image to be detected in the video to be detected, performs occlusion processing on the area except the area to be detected in the image to be detected, excludes the image of the non-detection area from the area that does not need to be detected, and obtains a target detection image, thereby highlighting the features of the target area that needs to be detected and reducing the amount of image data. The target detection image is input into a pre-trained package detection model to obtain a package detection image. The technical problems in the prior art that the detection efficiency is too slow and it is easy to scatter packages or detect packages incorrectly are solved, thereby improving the accuracy of image detection, reducing the probability of package loss and delay, and improving user satisfaction.

[0044] Embodiment 2

[0045] Figure 2 This is a flowchart of a package detection method provided in the second embodiment of the present invention. The relationship between this embodiment and the above embodiment can be further explained in detail in the specific training process of the package detection model. Figure 2 As shown, the method includes:

[0046] S210: Acquire a video to be detected, and extract an image to be detected from the video to be detected.

[0047] S220: performing occlusion processing on the area other than the to-be-detected area in the to-be-detected image to obtain a target detection image.

[0048] S230: Obtain a sample video, extract a sample image from the sample video, and perform occlusion processing on an area other than a detection area in the sample image to obtain a training image.

[0049] The sample video may be a video of the sorting process captured by a camera during parcel sorting.

[0050] Optionally, during the parcel sorting process, a sample shooting device can be used to shoot the area to be detected, and the video of the area to be detected shot by the sample shooting device is used as a sample video. Then, the image in the sample video is extracted as a sample image. For example, each frame of the sample video can be extracted as a sample image. After obtaining the sample image, the position information of the area to be detected in the sample image is determined, and then the area to be detected is delineated in the sample image, and then the area in the sample image except the detection area is blocked to obtain a training image. Among them, the method of blocking the area except the sample area can be the same as the processing method of the image to be detected, or it can be different. For example, the image can be blurred to block the area except the sample area.

[0051] Optionally, in another embodiment of the present invention, after extracting the sample image in the sample video and before performing occlusion processing on the area except the detection area in the sample image, the method further includes:

[0052] Perform image preprocessing on the sample images extracted from the sample video, wherein the image preprocessing includes at least one of screening processing, frame extraction processing, cleaning processing and data enhancement processing.

[0053] Optionally, the image screening process for the sample images extracted from the sample video may be to screen out sample image information whose clarity satisfies a preset clarity condition from all the sample images. For example, the sum of the pixel values ​​of each sample image may be calculated, and then the sample images whose pixel values ​​of the sample images are greater than a preset pixel value may be screened out.

[0054] Optionally, performing image frame extraction processing on the sample images extracted from the sample video may be to determine the sample images to be extracted in the sample video according to a preset frame extraction frequency.

[0055] Optionally, the image cleaning process for the sample images extracted from the sample video may be to detect all the sample images, detect problematic sample images with distorted screens, blurs, or defects, delete the problematic sample images, and then re-sort and rename the sample images.

[0056] Optionally, the performing image enhancement processing on the sample image extracted from the sample video includes at least one of the following operations:

[0057] Performing masking processing on the area of ​​the sample image except the area to be detected;

[0058] Cropping the sample image according to the position information of the area to be detected in the sample image, and enlarging the cropped image to enlarge the area to be detected;

[0059] A wrapping map is added to the area to be detected in the sample image.

[0060] Similarly, by performing occlusion processing on the area other than the to-be-detected area in the sample image, the package detection model can pay more attention to the packages in the to-be-detected area, thereby preventing the misidentification of normally sorted packages.

[0061] Optionally, after acquiring the sample image, the position information of the area to be detected in the sample image is determined, and then the area to be detected is delineated in the sample image, and then the area in the sample image other than the detection area is occluded to obtain the training image. The occlusion processing of the area other than the sample area may be blurring the image to occlude the area other than the sample area.

[0062] Optionally, after determining the corresponding coordinates of the area to be detected in the sample image, the coordinates of each pixel in the area of ​​the sample image except the area to be detected are determined according to the position information in the sample image and the corresponding coordinates of the area of ​​the sample image except the area to be detected, and image replacement is performed using the coordinates of each pixel in the area of ​​the sample image except the area to be detected to mask out the area except the area to be detected.

[0063] Taking into account that when the camera is shooting the video of the sorting area, affected by the shooting distance and the shooting angle, the display area of ​​the scattered packages far away from the camera is often relatively small in the image and not easy to be identified, and the real samples of scattered packages are relatively small, therefore, the area to be detected of the sample image can be cropped by information to obtain the sample image in a way of enlarging the area to be detected.

[0064] Optionally, obtain the position information of the area to be detected in the sample image, determine the pixel coordinates of the area to be detected in the sample image, determine the pixel coordinates of the area corresponding to each to-be-cropped coordinate of the area to be detected according to pre-set cropping coordinates, then crop the pixel points of the area corresponding to each cropping coordinate, and enlarge the cropped image to enlarge the area to be detected.

[0065] As mentioned above, in real sample videos, there are relatively few sample images containing spilled packages. In order to ensure the balance of positive and negative samples, the sample images with spilled packages can be increased by adding a package map in the area to be detected in the sample image, thereby improving the recognition accuracy of the package detection model for the spilled packages. The package map can be a package image of different shapes. Optionally, the package map can be a package that is cropped and left in the area to be detected in other images.

[0066] Optionally, other images including the area to be detected are obtained, and the wrapped images in the other images including the area to be detected are clipped as the wrapped maps, and then the wrapped maps are added to the area to be detected in the sample image. Exemplarily, adding the wrapped maps to the area to be detected in the sample image can replace some pixel values ​​in the area to be detected in the sample image with the pixel values ​​of the wrapped maps.

[0067] S240: Label the scattered packages in the training image to obtain a desired output image corresponding to the training image.

[0068] Specifically, after acquiring the training image, the training image is identified to identify the scattered packages in the training image, and then the scattered packages in the training image are labeled, and the training image after the scattered packages are labeled is used as the expected output image corresponding to the training image.

[0069] S250: Input the training image into a pre-established model to be trained to obtain a model detection image.

[0070] Among them, the model to be trained may include but is not limited to: any one of Convolutional Neural Networks (CNN), Fully Connected Neural Network (FCN), Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM).

[0071] Specifically, the training image is input into a pre-established model to be trained for training, and the model to be trained outputs a corresponding model detection image according to the input training image.

[0072] Optionally, the model detection image can be divided into four layers: input layer (input), neural network backbone network layer (Backbone), network layer (Neck), and prediction layer (Prediction). At the input end, the sample image is obtained, and the sample image Mosaic data is enhanced. Then, the adaptive anchor frame is calculated, and then the expected output image of the real frame is compared, and the loss of the two is calculated. Then, the back propagation update is performed according to the loss of the two, and the network parameters are iterated. The pictures from different shooting devices are uniformly scaled to a standard size and then sent to the detection network. In the neural network backbone network layer (Backbone), the image input sampling layer (Focus) structure adopts a slicing operation, and then undergoes a convolution operation of several convolution kernels, and finally becomes a feature map of uniform size. In the network layer (Neck), the CSP2 structure designed by the cross-stage local network (CSPNet) is adopted to enhance the ability of network feature fusion. In the prediction layer (Prediction), the loss calculation (GIOU_Loss) of the border prediction is used as the loss function of the model detection image.

[0073] S260. Adjust the model to be trained according to the detection loss between the model detection image and the expected output image to obtain a package detection model.

[0074] Specifically, the loss of the model detection image and the expected output image is calculated according to the preset loss function, the detection loss of the model detection image and the expected output image is determined, and then the to-be-trained model is adjusted according to the detection loss of the model detection image and the expected output image to obtain the package detection model. The preset loss function may be, but is not limited to, any one of a mean square loss function, a mean absolute value loss function, or a cross entropy loss function.

[0075] S270: Input the target detection image into a pre-trained package detection model to obtain a package detection image.

[0076] The technical solution of the embodiment of the present invention obtains the video to be detected, extracts the image to be detected in the video to be detected, performs occlusion processing on the area except the area to be detected in the image to be detected, excludes the image of the non-detection area from the area that does not need to be detected, and obtains the target detection image, thereby highlighting the characteristics of the target area that needs to be detected and reducing the amount of image data. When training the package detection model, the sample images are preprocessed to enhance the diversity of the sample images, reduce the impact of the problem sample images on the accuracy of model training, and eliminate the interference of the conveyor belt motion blur, reduce the amount of data for model training, and input the target detection image into the pre-trained package detection model to obtain the package detection image. The technical problems of the prior art that the detection efficiency is too slow and it is easy to scatter packages or detect packages incorrectly are solved, thereby improving the accuracy of image detection, reducing the probability of package loss and delay, and improving user satisfaction.

[0077] Embodiment 3

[0078] Figure 3 Flow chart of a parcel detection method provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments can be further described in detail the process of detecting a parcel spillage event in a parcel detection image. Figure 3 As shown, the method includes:

[0079] S310: Acquire a video to be detected, and extract an image to be detected from the video to be detected.

[0080] S320: performing occlusion processing on the area other than the to-be-detected area in the to-be-detected image to obtain a target detection image.

[0081] S330: Input the target detection image into a pre-trained package detection model to obtain a package detection image.

[0082] S340: issuing an early warning for the parcel spillage event according to the parcel detection image.

[0083] The parcel spilling event can be understood as an event based on the determination of the existence of parcel spilling in the parcel detection image. Specifically, when there is parcel spilling in the parcel detection image, it is determined that there is a parcel spilling event, and then an early warning of the parcel spilling event is issued.

[0084] Considering that in the actual parcel sorting process, there are situations where parcel spillage events are handled. In other words, parcel spillage events may be eliminated at any time. Therefore, when reporting parcel spillage events, the current frame of the image to be detected and the previous frame of the image to be detected can be combined to determine whether there is a parcel spillage event, and then, an early warning is issued through the parcel spillage event. Therefore, for the parcel spillage event, it is optional, in another embodiment of the present invention, the early warning of the parcel spillage event based on the parcel detection image includes:

[0085] If the previous frame of the video to be detected does not mark the spilled package and the package detection image corresponding to the current frame of the video to be detected is marked with a spilled package, an early warning of the package spillage event is issued; if the previous frame of the video to be detected and the package detection images corresponding to the current frame of the video to be detected are both marked with spilled packages, no early warning of the package spillage event is issued; if the previous frame of the video to be detected is marked with a spilled package and the package detection image corresponding to the current frame of the video to be detected is not marked with a spilled package, the early warning of the package spillage event is cancelled.

[0086] Specifically, when a spilled package is marked in the package detection image, the current frame image to be detected and the video to be detected in the video to be detected corresponding to the current package detection image are obtained, and then the package detection image of the image to be detected in the previous frame of the video to be detected is obtained. If the package detection image of the image to be detected in the previous frame of the video to be detected is not marked with a spilled package, it is considered that a package spillage event exists at this time, and an early warning of the package spillage event is issued; when a spilled package is marked in the package detection image, the current frame image to be detected and the video to be detected in the video to be detected corresponding to the current package detection image are obtained, and then the package detection image of the image to be detected in the previous frame of the video to be detected is obtained, such as If the package detection image of the previous frame of the image to be detected in the video to be detected is marked with a spilled package, it is considered that the current package spillage event has been warned. In order to prevent repeated warnings for the same package spillage event, no warning is given for the package spillage event. When the package detection image is not marked with a spilled package, the current frame of the image to be detected and the video to be detected corresponding to the current package detection image are obtained, and then the package detection image of the previous frame of the image to be detected in the video to be detected is obtained. If the package detection image of the previous frame of the image to be detected in the video to be detected is marked with a spilled package, the package spillage event has been resolved at this time, and the warning for the package spillage event is canceled.

[0087] Optionally, if a preset number of images to be detected marked with spilled packages are detected in the video to be detected, and the images to be detected are continuous in time sequence, an early warning is issued for the spilled package event.

[0088] Specifically, when a parcel is marked with a spilled parcel in the parcel detection image, the current frame of the image to be detected and the video to be detected in the video to be detected corresponding to the current parcel detection image are obtained, and then the parcel detection image of the image to be detected in the video to be detected is obtained. If the parcel detection images of multiple frames of images to be detected that are consecutive in time sequence in the video to be detected are all marked with a spilled parcel, it is considered that the current parcel spill event has not been handled, and it is necessary to issue an early warning for the current parcel spill event again. Among them, the parcel detection images of multiple frames of images to be detected that are consecutive in time sequence can be the parcel detection images of multiple frames of images to be detected that are consecutive in arrangement order after being arranged according to the time sequence of the parcel detection images.

[0089] Optionally, after determining that a warning is required for a package spillage event, a package detection image of the current package spillage event is obtained, and the time information of the frame where the package detection image is located, the shooting device identifier corresponding to the video to be detected corresponding to the package detection image, and the number of spilled packages marked in the package detection image are determined; the time information, the shooting device identifier and the number of packages are reported as event reporting information, so as to issue a warning based on the event reporting information.

[0090] Optionally, the time of each frame of the package detection image is used as the dropping time of the spilled package, the coordinate position information is determined by the identifier of the shooting device of the video to be detected corresponding to the package detection image, and the number of spilled packages marked in the package detection image is used as the number of missed packages. Then, the dropping time, coordinate position information and number of missed packages of the spilled package incident are stored, and it is determined whether the package spillage incident needs to be reported according to the reporting logic.

[0091] Optionally, after an early warning is issued when it is determined that a package has been spilled, a corresponding package spill push message is determined according to the package spill event, and the package spill push message is pushed to the management end, thereby reminding the management end to handle the package spill event.

[0092] The technical solution of the embodiment of the present invention obtains the video to be detected, extracts the image to be detected in the video to be detected, performs occlusion processing on the area except the area to be detected in the image to be detected, excludes the image of the non-detection area from the area that does not need to be detected, and obtains the target detection image, thereby highlighting the characteristics of the target area that needs to be detected and reducing the amount of image data. The target detection image is input into the pre-trained package detection model to obtain the package detection image, and the package spillage event is jointly determined based on the package detection image and the previous frame package detection image. The early warning of the package spillage event is reported and cancelled in real time, reducing the misjudgment of package loss and improving the accuracy of the judgment of package spillage. The technical problems of too slow detection efficiency and easy to scatter packages or detect wrong packages in the prior art are solved, thereby improving the accuracy of image detection, reducing the probability of package loss and delay, and improving user satisfaction.

[0093] Embodiment 4

[0094] Figure 4 This is a schematic diagram of the structure of a package detection device provided by Embodiment 4 of the present invention. Figure 4 As shown, the device specifically includes: an image extraction module 410, an image processing module 420 and a package detection module 430. Among them:

[0095] The image extraction module 410 is used to obtain a video to be detected and extract an image to be detected from the video to be detected;

[0096] The image processing module 420 is used to perform occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image;

[0097] The package detection module 430 is used to input the target detection image into a pre-trained package detection model to obtain a package detection image.

[0098] The technical solution of the embodiment of the present invention obtains a video to be detected, extracts an image to be detected in the video to be detected, performs occlusion processing on the area except the area to be detected in the image to be detected, excludes the image of the non-detection area from the area that does not need to be detected, and obtains a target detection image, thereby highlighting the features of the target area that needs to be detected and reducing the amount of image data. The target detection image is input into a pre-trained package detection model to obtain a package detection image. The technical problems in the prior art that the detection efficiency is too slow and it is easy to scatter packages or detect packages incorrectly are solved, thereby improving the accuracy of image detection, reducing the probability of package loss and delay, and improving user satisfaction.

[0099] Optionally, the image extraction module 410 is specifically used for:

[0100] Extracting the image to be detected from the video to be detected according to a preset image extraction frequency;

[0101] Randomly extracting a preset number of images to be detected from the video to be detected;

[0102] Extracting the image to be detected from the video to be detected according to a preset extraction time interval;

[0103] Each frame of the image in the video to be detected is extracted as an image to be detected.

[0104] Optionally, the image processing module 420 is specifically used for:

[0105] Based on preset coordinates corresponding to the area to be detected in the image to be detected, an occlusion process is performed on the area other than the detection area in the image to be detected.

[0106] Optionally, before the image processing module 420, the device is further configured to:

[0107] Acquire a sample video, extract a sample image from the sample video, and perform occlusion processing on an area other than a detection area in the sample image to obtain a training image;

[0108] Annotating the scattered packages in the training image to obtain a desired output image corresponding to the training image;

[0109] Inputting the training image into a pre-established model to be trained to obtain a model detection image;

[0110] The model to be trained is adjusted according to the detection loss between the model detection image and the expected output image to obtain a package detection model.

[0111] Optionally, before the image processing module 420, the device is further configured to:

[0112] Perform image preprocessing on the sample images extracted from the sample video, wherein the image preprocessing includes at least one of screening processing, frame extraction processing, cleaning processing and data enhancement processing.

[0113] Optionally, before the image processing module 420, the device is further configured to:

[0114] Performing masking processing on the area of ​​the sample image except the area to be detected;

[0115] Cropping the sample image according to the position information of the area to be detected in the sample image, and enlarging the cropped image to enlarge the area to be detected;

[0116] A wrapping map is added to the area to be detected in the sample image.

[0117] Optionally, after the package detection module 430, the device is further configured to:

[0118] An early warning is issued for the parcel spillage event based on the parcel detection image.

[0119] Optionally, after the package detection module 430, the device is further configured to:

[0120] If no spilled package is marked in the previous frame of the image to be detected in the video to be detected and a spilled package is marked in the package detection image corresponding to the current frame of the image to be detected, an early warning is issued for the package spilling event;

[0121] If the package detection images corresponding to the previous frame of the image to be detected and the current frame of the image to be detected in the video to be detected both have the package spilled marked, no warning is issued for the package spilling event;

[0122] If no package spillage is marked in the package detection images corresponding to the previous frame of the image to be detected and the current frame of the image to be detected in the video to be detected, no warning is issued for the package spillage event;

[0123] If a previous frame of the image to be detected in the video to be detected is marked with a spilled package, and a package detection image corresponding to the current frame of the image to be detected is not marked with a spilled package, the warning for the package spillage event is canceled.

[0124] Optionally, after the package detection module 430, the device is further configured to:

[0125] If a preset number of images to be detected marked with spilled packages are detected in the video to be detected, and the images to be detected are continuous in time sequence, an early warning is issued for the spilled package event.

[0126] The package detection device provided in the embodiment of the present invention can execute the package detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the package detection method.

[0127] It is worth noting that in the embodiment of the above-mentioned package detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0128] Embodiment 5

[0129] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0130] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a package detection method.

[0133] In some embodiments, the package detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the package detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the package detection method in any other appropriate manner (e.g., by means of firmware).

[0134] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0136] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0141] Embodiment 6

[0142] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the package detection method provided in any embodiment of the present invention are implemented. The method includes:

[0143] Acquire a video to be detected, and extract an image to be detected from the video to be detected;

[0144] Performing occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image;

[0145] The target detection image is input into a pre-trained package detection model to obtain a package detection image.

[0146] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0147] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0148] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0149] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0150] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0151] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the attached claims. The above specific implementation methods do not constitute a limitation on the scope of protection of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A package detection method, characterized in that: include: Acquire a sample video, extract a sample image from the sample video, and perform occlusion processing on an area other than a detection area in the sample image to obtain a training image; Annotating the scattered packages in the training image to obtain a desired output image corresponding to the training image; Inputting the training image into a pre-established model to be trained to obtain a model detection image; Adjusting the model to be trained according to the detection loss between the model detection image and the expected output image to obtain a package detection model; Acquire a video to be detected, and extract an image to be detected from the video to be detected; Performing occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image; The target detection image is input into a pre-trained package detection model to obtain a package detection image.

2. The method according to claim 1, characterized in that: The extracting the image to be detected from the video to be detected includes at least one of the following operations: Extracting the image to be detected from the video to be detected according to a preset image extraction frequency; Randomly extracting a preset number of images to be detected from the video to be detected; Extracting the image to be detected from the video to be detected according to a preset extraction time interval; Each frame of the image in the video to be detected is extracted as an image to be detected.

3. The method according to claim 1, characterized in that The performing occlusion processing on the area other than the detection area in the image to be detected includes: Based on preset coordinates corresponding to the area to be detected in the image to be detected, an occlusion process is performed on the area other than the detection area in the image to be detected.

4. The method according to claim 1, characterized in that: After extracting the sample image from the sample video and before performing the occlusion processing on the area except the detection area in the sample image, the method further includes: Perform image preprocessing on the sample images extracted from the sample video, wherein the image preprocessing includes at least one of screening processing, frame extraction processing, cleaning processing and data enhancement processing.

5. The method according to claim 4, characterized in that The performing image enhancement processing on the sample image extracted from the sample video includes at least one of the following operations: Performing masking processing on the area of ​​the sample image except the area to be detected; Cropping the sample image according to the position information of the area to be detected in the sample image, and enlarging the cropped image to enlarge the area to be detected; A wrapping map is added to the area to be detected in the sample image.

6. The method according to claim 1, characterized in that After obtaining the package detection image, the method further includes: An early warning is issued for a parcel spillage event based on the parcel detection image.

7. The method according to claim 6, characterized in that The step of issuing an early warning for the parcel spillage event according to the parcel detection image includes: If no spilled package is marked in the previous frame of the image to be detected in the video to be detected and a spilled package is marked in the package detection image corresponding to the current frame of the image to be detected, an early warning is issued for the package spilling event; If the package detection images corresponding to the previous frame of the image to be detected and the current frame of the image to be detected in the video to be detected both have the package spilled marked, no warning is issued for the package spilling event; If no package spillage is marked in the package detection images corresponding to the previous frame of the image to be detected and the current frame of the image to be detected in the video to be detected, no warning is issued for the package spillage event; If a previous frame of the image to be detected in the video to be detected is marked with a spilled package, and a package detection image corresponding to the current frame of the image to be detected is not marked with a spilled package, the warning for the package spillage event is canceled.

8. The method according to claim 7, characterized in that Also includes: If a preset number of images to be detected marked with spilled packages are detected in the video to be detected, and the images to be detected are continuous in time sequence, an early warning is issued for the spilled package event.

9. A package detection device, characterized in that: include: Acquire a sample video, extract a sample image from the sample video, and perform occlusion processing on an area other than a detection area in the sample image to obtain a training image; Annotating the scattered packages in the training image to obtain a desired output image corresponding to the training image; Inputting the training image into a pre-established model to be trained to obtain a model detection image; Adjusting the model to be trained according to the detection loss between the model detection image and the expected output image to obtain a package detection model; An image extraction module is used to obtain a video to be detected and extract an image to be detected from the video to be detected; An image processing module is used to perform occlusion processing on the area other than the area to be detected in the image to be detected to obtain a target detection image; The package detection module is used to input the target detection image into a pre-trained package detection model to obtain a package detection image.

Images