Automatic Target Object Detection System Based on Image Processing

Abstract

The invention discloses an automatic target object detection system based on image processing, which belongs to the technical field of image processing. The system stores a package on a shelf, adds an image collected from the surface where the package label is located and a two-dimensional code storing basic information to obtain a fused identification surface. After the package is placed on the shelf, if a user initiates a pickup request, the pickup surface of the corresponding recipient is obtained according to the identity of the initiator, and the position of the corresponding package on the shelf can be accurately identified according to the pickup surface and four margins; and a picture of the real-time position can be displayed, and the package position is automatically noted in the picture. This process can basically be achieved without the assistance of other personnel throughout the process, and the package can be quickly located and searched through image processing and analysis, without the need to identify and pick up according to the pickup code, which is convenient, fast, and energy-saving.

Description

Technical Field

[0001] The invention belongs to the field of image processing and analysis, and in particular relates to an automatic target object detection system based on image processing. Background Art

[0002] For cargo transportation, delivery and other businesses, due to the large volume of business and the large number of packages, the processing of orders also presents the characteristics of "multiple varieties, small batches, and multiple batches". In particular, the introduction of the "new retail" concept has put forward higher requirements for the intelligence of warehousing systems. In order to process packages quickly with high efficiency and low cost, an automated and intelligent image recognition system is particularly important.

[0003] The intelligent logistics solution is extremely flexible and can also be used between multiple logistics points according to the business volume, which cannot be replaced by the traditional model. For example, the patent with the publication number CN117765065A discloses a single-piece separated package rapid positioning method based on target detection, which relates to the field of logistics technology. The method collects the RGB image to be detected and the depth data to be detected of the single-piece separation area through an RGBD camera located above the single-piece separation area, and then uses the lightweight target detection model based on YOLO to perform target detection and extract the target detection frame of each package; then determine the reference depth value of the target detection frame according to the depth data to be detected in each target detection frame, and obtain the projection coordinates of the four vertices of the target detection frame on the separation belt according to the reference depth value of each target detection frame, so as to determine the location information of the corresponding package. This method can locate the location information of the package in the separation area in real time and quickly, has low requirements on the quality of depth data, small amount of calculation, and high operating efficiency, which can improve the efficiency of logistics sorting and processing and meet the needs of practical applications.

[0004] Also, Chinese patents CN117974768A, CN117593358A, CN112308915A, and the like, all provide a way to locate a package. However, for the process of picking up express packages, the express stations in the prior art all use manual sorting and labeling (such as the existing Cainiao Station, Duoduo Maicai and other express stations, which use a method such as 7-2-2024 to place the package on shelf No. 7, package No. 2024 on the second floor, and go to see someone to pick up the goods according to this pickup code), and classify them by labels and place them on different shelves, so as to achieve the pickup of a large number of express deliveries. This placement process is too cumbersome and requires printing paper again, which adds costs and causes a certain amount of resource waste. In order to achieve rapid automatic detection of target objects, guide users to pick up express deliveries, realize a "paperless" automatic detection process of target objects, and save energy, the present application specifically provides an automatic detection system for target objects based on image processing. Summary of the invention

[0005] The object of the present invention is to provide an automatic target object detection system based on image processing.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The automatic detection system of target objects based on image processing includes:

[0008] The automatic target object detection system based on image processing is characterized by comprising:

[0009] An image acquisition module automatically acquires basic information of the package and simultaneously determines the four margins of the package based on the distance between the label on the package and the four edge lines of the package;

[0010] The information fusion module retrieves the image of the surface where the package label is located from the image acquisition module, and fuses it with the QR code storing the basic information to obtain the fused identification surface. The identification surface and the four margins are fused to form the comparison information of the package;

[0011] An information initiation module is used for a user to initiate a pickup request. If a user initiates a pickup request, the identification face of the corresponding recipient is obtained according to the user's identity information, and the identification face is marked as the pickup face;

[0012] An image processing module captures a real-time analysis image of the package containing the label from the video obtained by the image acquisition module, and compares the pickup surface with the real-time analysis image. If the real-time analysis image of any package has the highest similarity with the pickup surface and the four margins are consistent, the corresponding package location is marked in the image, and the package location image containing the shelf is marked as a real-time pickup image;

[0013] Processor, which sends real-time pickup images to smart devices for display.

[0014] Beneficial effects of the present invention:

[0015] The present invention stores the parcel on the shelf, adds the image collected from the surface where the parcel label is located and the QR code storing the basic information to obtain the fused identification surface. After the parcel is placed on the shelf, if the user initiates a pickup request, the pickup surface of the corresponding recipient is obtained according to the identity of the initiator, and the position of the corresponding parcel on the shelf can be accurately identified according to the pickup surface and the four margins, so as to realize automatic detection and positioning of the target based on image processing, and directly transmit the parcel positioning information to the smart device for display;

[0016] It can also display a picture of the real-time location, with the package location automatically noted in the picture. This process can basically be accomplished without the assistance of other personnel, and the package can be found quickly without having to identify and pick up the package based on the pickup code. It is convenient, fast, and saves energy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0018] Figure 1 A diagram of an automatic target object detection system based on image processing according to the present invention;

[0019] Figure 2 A schematic diagram of a field environment for implementing the method of the present invention;

[0020] Figure 3 A partial schematic diagram of the method of the present invention in which the parcel of the express locker has not been taken away;

[0021] Figure 4 for Figure 3 Schematic diagram of package removal for comparison. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0023] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0024] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0026] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0028] Embodiment 1:

[0029] The present application is used to provide an automatic target object detection system based on image processing, such as Figure 1 As shown, it includes an image acquisition module, an information fusion module, an information initiation module, an image processing module, a processor, and a pickup confirmation module, wherein the image acquisition module is connected to the processor in communication, the information fusion module is connected to the processor in communication, the information initiation module is connected to the processor in communication, the image processing module is connected to the processor in communication, and the pickup confirmation module is connected to the processor in communication; specifically:

[0030] An image acquisition module, which automatically acquires basic information of the parcel when the parcel enters the transfer point, including the sender, recipient and logistics details, and simultaneously determines the four margins of the parcel according to the distance between the center of the label on the parcel and the four edge lines, and transmits the acquired information to the processor;

[0031] Information fusion module: After the package is stored in the shelf, the information fusion module retrieves the image of the surface where the package label is located from the image acquisition module, and fuses it with the QR code that stores the basic information to obtain the fused identification surface. The identification surface and the four margins are fused to form the comparison information of the package;

[0032] An information initiation module, which is used by a user to initiate a pickup request. After a package is placed on the shelf, if the user initiates a pickup request, the identification face of the corresponding recipient is obtained according to the user's identity information, and the identification face is marked as the pickup face;

[0033] An image processing module, wherein the image processing module captures a real-time analysis image of the package including the face where the label is located from the video acquired by the image acquisition module in real time, and compares the pickup face with the real-time analysis image. If the real-time analysis image of any package has the highest similarity with the pickup face and the four margins are consistent, the corresponding package location is marked in the image, and the package location image including the shelf is marked as a real-time pickup image;

[0034] Processor, which sends real-time pickup images to smart devices for display.

[0035] As a first embodiment of the present application, a target object automatic detection system based on image processing performs the following steps when processing and locating a package image:

[0036] Step 1: When a package enters the transfer point, in addition to scanning the label on the package to confirm its identity, detailed information of the package will be collected at the same time. Of course, the identity determination here refers to scanning the barcode with a barcode scanner to determine the shipping information, receiving information and logistics information of the corresponding package. The shipping information includes the sender and shipping address of the corresponding package, the receiving information includes the recipient and receiving address of the corresponding package, and the logistics information includes the shipping process monitoring of the corresponding package, where it is sent from, which intermediate station it arrives at, and who delivers it to the transfer point; the specific methods for collecting detailed information are:

[0037] Get the label side, which is the side of the package that has the label attached to it;

[0038] Automatically obtain the four margins of the label from the edge of the surface where the label is located. The four margins refer to the shortest distance between the center point of the label and the four edge lines of the surface where the label is located, which is actually the vertical distance. The four shortest distances are marked as the four margins;

[0039] Then get the image of the face where the label is located, mark it as the basic face, and process the basic face. The specific processing method is as follows:

[0040] Get the fundamentals;

[0041] Automatically obtain the shipping information and receiving information, and automatically store the text in the shipping information and receiving information in the QR code in the form of a QR code according to the QR code forming technology;

[0042] Mark the QR code stored in the shipping information and the receiving information as an identification code;

[0043] Then the identification code is assigned to the basic surface of the corresponding package, and the basic surface assigned with the identification code is re-labeled as the identification surface;

[0044] Obtain the processed identification surface, and fuse the identification surface and the four margins to form the comparison information of the corresponding package;

[0045] Step 2: After collecting detailed information, enter the middleware settings. The specific method of middleware settings is:

[0046] Automatically place the parcels on the shelves at the transfer point. When all parcels are placed on the shelves, they are automatically placed with the label facing outwards. Here, outwards means that the label side will not be blocked by the shelves or other parcels, and the label side can be captured by the camera set at the specified position.

[0047] As an embodiment provided by the present invention, preferably, the package is transferred to the shelf by an automatic conveyor belt, and the height of one end of the conveyor belt is adjustable to adapt to shelves of different heights (different layer heights); the package is placed with the label side facing outward by a robot and a camera on the conveyor belt; this is a prior art and will not be described in detail; alternatively, the package is placed manually with the label side facing outward.

[0048] As an embodiment provided by the present invention, preferably, Figure 2 As shown, a camera is set to ensure that the camera can monitor the label surface of all packages, so as to obtain real-time video of the label surface of all packages;

[0049] After setting up the camera, the basic conversion ratio will be automatically obtained. The specific method of obtaining the basic conversion ratio is as follows:

[0050] First, a specified object is obtained in the camera monitoring area. The length of the specified object is known, and the number of pixels occupied by the length of the specified object is obtained synchronously;

[0051] Divide the known length by the number of pixels occupied, and the resulting value is marked as the unit length;

[0052] Step 3: After completing the above work and the preparation work is done, any person can initiate a package and lock the location of the package through automatic analysis, and display it intuitively for the person to pick it up directly; the specific method of automatic analysis is:

[0053] St1: When the signal is acquired, all the parcels of the corresponding initiator as the recipient will be automatically acquired. The acquisition method is to determine the sender and recipient information stored in the QR code in the identification surface, obtain all the identification surfaces of the corresponding initiator's parcels, and mark them as pickup surfaces;

[0054] St2: After that, the camera obtains the real-time video of the label side of all packages, automatically captures a frame of the highest definition photo, and then marks the photos of the label side of all packages as real-time analysis graphs;

[0055] St3: Compare the pickup face with all the real-time analysis graphs to obtain the similarity between the pickup face and all the real-time analysis graphs, and mark the real-time analysis graph with the highest similarity as the graph to be confirmed;

[0056] St4: Then obtain the real-time four-side distances in the to-be-confirmed image, obtain the pixel points required for the center of the label in the image to reach the four edge lines, multiply it by the unit length to get the corresponding distance, compare the real-time four-side distances with the four-side distances corresponding to the pickup surface, and if the data are consistent, generate a confirmation signal, indicating that the corresponding package is found, automatically obtain the real-time photo of the shelf where the corresponding package is located, and mark it as a real-time pickup photo. After obtaining all the real-time pickup photos of the initiator, they will be automatically displayed on the smart device, so that the initiator can quickly pick up the express on the shelf;

[0057] St5: If the real-time four-sided distances are inconsistent with the corresponding four-sided distances, repeat steps St2-St4; if the number of repetitions is greater than or equal to four times and the corresponding real-time pickup photo has not been found, the courier station manager will be automatically notified for processing.

[0058] Embodiment 2:

[0059] As the second embodiment of the present invention, this application is implemented on the basis of the first embodiment, and is different from the first embodiment in that it also includes a pickup confirmation module, which is used for pickup confirmation. After obtaining all the real-time pickup photos of the initiator, this application will automatically display them on the smart device, and is also used to monitor the pickup process. The specific monitoring method is:

[0060] When any real-time pickup photo is detected, the location of the corresponding real-time pickup photo and the image of the location will be synchronously obtained, and the image will be marked as a foreground image;

[0061] Afterwards, when it is detected that a person appears at the location and then disappears again, an image of the location is automatically acquired and marked as a background photo;

[0062] like Figure 3 and Figure 4 As shown:

[0063] Mark the differences between the background image and the foreground image, automatically obtain the area of ​​the difference, and mark it as the difference area;

[0064] Then, the area occupied by the package in the real-time pickup photo is obtained and marked as the actual area;

[0065] When the actual area is ≤ the difference area ≤ XX1*actual area, and there is no package at the location of the package in the corresponding real-time pickup photo, it means that the corresponding package has been taken away, and the arrival information of the package will be automatically updated;

[0066] The update completion means that the corresponding time of the package taken away has ended and no subsequent monitoring is required; here X1 is a preset value, generally 1.5.

[0067] Embodiment three:

[0068] As the third embodiment of the present invention, this application is implemented on the basis of the second embodiment, and the difference from the second embodiment is that in this embodiment, a low-cargo monitoring module is further included, and the low-cargo monitoring module is used for low-cargo monitoring, and the specific method is as follows:

[0069] When there are no new express deliveries, the real-life photos taken by the camera are used to automatically identify the number of packages in the real-life photos;

[0070] Periodically obtain the current number of packages, and the interval between cycles is preset by the administrator;

[0071] When the number of parcels detected in any period is less than the number of parcels at the node of the previous period, the reduced number will be marked as the reduced number; then the number of parcels to be picked up in this period will be automatically obtained, and the number of parcels to be picked up will be consistent with the number of real-time pickup photos;

[0072] If the number of reductions is equal to the number of items to be picked up, no processing will be done. If the number of reductions is greater than the number of items to be picked up, the corresponding location and image of any real-time pickup photo generated during this cycle will be automatically obtained, and the image will be marked as the foreground image.

[0073] Afterwards, when it is detected that a person appears at the location and then disappears again, an image of the location is automatically acquired and marked as a background photo;

[0074] like Figure 3 and Figure 4 As shown, the differences between the background image and the foreground image are marked, and the area of ​​the difference part is automatically obtained and marked as the difference area;

[0075] Then, the area occupied by the package in the real-time pickup photo is obtained and marked as the actual area;

[0076] When the difference area is greater than XX1*actual package area, and there is no package at the location of the package in the corresponding real-time pickup photo, a video of someone picking up the package in the real-time pickup photo will be automatically obtained and retained as evidence video;

[0077] Here XX1 is a preset value, which is generally 1.5.

[0078] Embodiment 4:

[0079] As the fourth embodiment of the present invention, this embodiment is implemented on the basis of the third embodiment. The difference from the third embodiment is that this embodiment provides a method for identifying the number of packages in a real-life photo. The specific method is as follows:

[0080] Establish a neural network recognition model, which can be specifically based on the CNN algorithm or other artificial recognition algorithms capable of identifying express delivery labels;

[0081] Construct a data sample that contains several identical or different express labels and pass it as input to the recognition model for recognition training. After the training is completed;

[0082] Obtain several test sets with known results. The test sets are courier labels different from the data samples. Input the test sets into the recognition model, output the recognition results, and automatically obtain the recognition success rate. When the success rate exceeds X2, mark the recognition model as a standard recognition model. Otherwise, re-obtain several data samples for re-recognition and then test until a standard recognition model is generated.

[0083] The standard recognition model is used to obtain the number of express labels in the real-life photo and mark it as the number of parcels.

[0084] Embodiment five:

[0085] This embodiment is used to integrate the first to fourth embodiments for implementation.

[0086] The present application also provides a method for decoding information on multi-sided distributed dense labels on a package. The information decoding method uses the aforementioned visual feature positioning method to decode user pickup information, making it easier for users to quickly find express deliveries.

[0087] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. An automatic target object detection system based on image processing, characterized in that: include: An image acquisition module automatically acquires basic information of the package and simultaneously determines the four margins of the package based on the distance between the label on the package and the four edge lines of the package; The information fusion module retrieves the image of the surface where the package label is located from the image acquisition module, and fuses it with the QR code storing the basic information to obtain the fused identification surface. When obtaining the identification surface, the following steps are performed: The image of the surface where the label is located is obtained and marked as the basic surface; the mailing information and the receiving information are automatically obtained, and the text in the mailing information and the receiving information is stored in the QR code in the form of a QR code according to the QR code forming technology; the QR code storing the mailing information and the receiving information is marked as an identification code; the identification code is assigned to the basic surface of the corresponding package, and the basic surface assigned with the identification code is marked as the identification surface; The recognition surface and the four margins are fused to form the comparison information of the package; An information initiation module is used for a user to initiate a pickup request. If a user initiates a pickup request, the identification face of the corresponding recipient is obtained according to the user's identity information, and the identification face is marked as the pickup face; An image processing module captures a real-time analysis image of the package containing the label from the video obtained by the image acquisition module, and compares the pickup surface with the real-time analysis image. If the real-time analysis image of any package has the highest similarity with the pickup surface and the four margins are consistent, the corresponding package location is marked in the image, and the package location image containing the shelf is marked as a real-time pickup image; Processor, which sends real-time pickup images to smart devices for display.

2. The automatic target object detection system based on image processing according to claim 1 is characterized in that: The basic information includes the sender, the recipient and the logistics details. The image acquisition module performs the following steps when acquiring the four margins: Get the label and the image of the surface where the label is located; Extract the center point of the label; The four margins are the shortest distances from the center point to the four edge lines of the surface.

3. The automatic target object detection system based on image processing according to claim 1 is characterized in that: When the image acquisition module acquires the package image: When placing a package on a shelf, keep the label side of the package facing outwards. Facing outwards means that the label side will not be blocked by the shelf or other packages. The image acquisition module collects the package image through a camera set at a specified position.

4. The automatic target object detection system based on image processing according to claim 1 is characterized in that: The image acquisition module performs the following steps when acquiring the distance value: A designated object is acquired in the camera monitoring area, the length of the designated object is known, and the number of pixels occupied by the length of the designated object is acquired simultaneously; Divide the known length by the number of pixels occupied, and the resulting value is marked as the unit length; When you need to obtain the length of any object later, you only need to obtain the number of pixels where the side length of the corresponding object is located, and multiply the number of pixels by the unit length to get the length of the corresponding object.

5. The automatic target object detection system based on image processing according to claim 1 is characterized in that: The specific method of the image processing module to obtain the real-time pickup image is: St1: When any person initiates a pickup signal, all the packages of the corresponding initiator as the recipient are automatically obtained through the sender and recipient information stored in the QR code in the identification surface, and all the identification surfaces of the corresponding initiator's packages are obtained and marked as the pickup surface; St2: After that, the camera obtains the real-time video of the label side of all packages, automatically captures a frame of the highest definition photo, and marks the photos of the label side of all packages as real-time analysis graphs; St3: Compare the pickup face with all the real-time analysis graphs to obtain the similarity between the pickup face and all the real-time analysis graphs, and mark the real-time analysis graph with the highest similarity as the graph to be confirmed; St4: Then obtain the real-time four-side distances in the image to be confirmed, obtain the pixel points required for the center of the label in the image to reach the four edge lines, multiply it by the unit length to get the corresponding distance, compare the real-time four-side distances with the four-side distances corresponding to the pickup surface, and if the data are consistent, generate a confirmation signal, indicating that the corresponding package is found, automatically obtain the real-time photo of the shelf where the corresponding package is located, mark it as a real-time pickup photo, and obtain all the real-time pickup photos of the initiator, which will be automatically displayed on the smart device; St5: If the real-time four-sided distances are inconsistent with the corresponding four-sided distances, repeat steps St2-St4; if the number of repetitions is greater than or equal to four times and the corresponding real-time pickup photo has not been found, the courier station manager is automatically notified for processing.

6. The automatic target object detection system based on image processing according to claim 1, characterized in that: It also includes a pickup confirmation module, which is used for pickup confirmation. The pickup confirmation module performs the following steps when performing pickup confirmation: When any real-time pickup photo is detected, the location of the corresponding real-time pickup photo and the image of the location will be synchronously obtained, and the image will be marked as a foreground image; Afterwards, when it is detected that a person appears at the location and then disappears again, an image of the location is automatically acquired and marked as a background photo; Mark the differences between the background image and the foreground image, automatically obtain the area of ​​the difference, and mark it as the difference area; then obtain the area occupied by the package in the real-time pickup photo and mark it as the actual area; When the actual area ≤ the difference area ≤ X1*actual area, and there is no package at the location of the package in the corresponding real-time pickup photo, it means that the corresponding package has been taken away. At this time, the arrival information of the package is automatically updated. The update completion means that the corresponding time of the taken package has ended and no subsequent monitoring is required; here X1 is a preset value.

7. The automatic target object detection system based on image processing according to claim 1, characterized in that: The low-cargo monitoring module is further included, and the low-cargo monitoring module is used for low-cargo monitoring. When the low-cargo monitoring module performs low-cargo monitoring, the following steps are performed: When there are no new express deliveries, the real-life photos taken by the camera are used to automatically identify the number of packages in the real-life photos; Periodically obtain the current number of packages, and the interval between cycles is preset by the administrator; When the number of parcels detected in any period is less than the number of parcels at the node of the previous period, the reduced number will be marked as the reduced number; then the number of parcels to be picked up in this period will be automatically obtained, and the number of parcels to be picked up will be consistent with the number of real-time pickup photos; If the number of reductions is equal to the number of items to be picked up, no processing will be done. If the number of reductions is greater than the number of items to be picked up, the corresponding location and image of any real-time pickup photo generated during this cycle will be automatically obtained, and the image will be marked as the foreground image. Afterwards, when it is detected that a person appears at the location and then disappears again, an image of the location is automatically acquired and marked as a background photo; Mark the differences between the background image and the foreground image, automatically obtain the area of ​​the difference, and mark it as the difference area; then obtain the area occupied by the package in the real-time pickup photo and mark it as the actual area; When the difference area is greater than XX1*actual item area, and there is no package at the location of the package in the corresponding real-time pickup photo, a video of someone picking up the package in the real-time pickup photo will be automatically obtained and retained as an evidence video; here XX1 is a preset value.

Images