Logistics operation state detection method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a logistics operation state detection method and device, electronic equipment, a storage medium and a program product, and can be applied to the technical field of computers and the field of warehouse logistics, and the method comprises the steps: obtaining a conveying line operation image; target detection is carried out on the conveying line operation image, a logistics object transported by the conveying line is obtained, and the conveying line operation image comprises a specified conveying line operation area; and the logistics operation state of the conveying line is detected based on the position relation between the logistics object and the operation area of the conveying line.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology and warehousing and logistics, and more specifically, to a logistics operation status detection method, device, electronic equipment, storage medium and program product. Background Art

[0002] With the rapid development of warehousing and logistics technology, intelligent logistics equipment such as conveyor lines and handling robots will be used in warehousing and logistics operations to complete logistics operations such as picking and transportation of goods, so as to improve the efficiency of warehousing and logistics operations.

[0003] In the process of realizing the concept disclosed herein, the inventors discovered that there are at least the following problems in the related art: the operating status of logistics equipment is difficult to detect accurately during operation, resulting in low logistics efficiency of goods or even damage and loss of goods. Summary of the invention

[0004] In view of this, the present disclosure provides a logistics operation status detection method, device, electronic device, storage medium and program product.

[0005] One aspect of the present disclosure provides a method for detecting a logistics operation status, comprising: acquiring a conveyor line operation image; performing target detection on the conveyor line operation image to obtain a logistics object transported by the conveyor line, wherein the conveyor line operation image includes a designated conveyor line operation area; and detecting the logistics operation status of the conveyor line based on a positional relationship between the logistics object and the conveyor line operation area.

[0006] According to an embodiment of the present disclosure, based on the positional relationship between a logistics object and an operating area of ​​a conveyor line, detecting the logistics operating status of a conveyor line includes: determining the duration of a logistics object's stay in the operating area of ​​a conveyor line according to the positional relationship; and generating an abnormality detection result characterizing an abnormality in the logistics operating status of the conveyor line in response to a stay duration of the logistics object being greater than or equal to a first preset duration threshold.

[0007] According to an embodiment of the present disclosure, in response to a logistics object's residence time being greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line includes: in response to a logistics object's residence time being greater than or equal to a first preset time threshold, determining a size ratio relationship between an object size of a first logistics object and an area size of an operating area of ​​the conveyor line, the first logistics object being a logistics object located in the operating area of ​​the conveyor line; in response to the size ratio relationship satisfying a preset size ratio condition, generating an object congestion detection result, the abnormality detection result including an object congestion detection result.

[0008] According to an embodiment of the present disclosure, in response to a logistics object staying time greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line includes: in response to a logistics object staying time greater than or equal to a second preset time threshold, generating an object congestion detection result, wherein the time represented by the first preset time threshold is greater than the time represented by the second preset time threshold.

[0009] According to an embodiment of the present disclosure, in response to a logistics object's residence time being greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line also includes: in response to at least one logistics object's residence time being greater than or equal to the first preset time threshold, the object residence time of all logistics objects in the conveyor line operation area being greater than a second preset time threshold, and the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not satisfy a preset size ratio condition, generating an object card detection result, wherein the abnormality detection result includes an object card detection result, and the first logistics object is a logistics object staying in the conveyor line operation area.

[0010] According to an embodiment of the present disclosure, based on the positional relationship between the logistics objects and the conveyor line operating area, detecting the logistics operating status of the conveyor line also includes: determining a congestion index parameter characterizing the object congestion situation of the conveyor line according to a size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operating area, wherein the first logistics object is a logistics object located in the conveyor line operating area; and sending the congestion index parameter to a designated client.

[0011] According to an embodiment of the present disclosure, a conveyor line operation image includes multiple frames; wherein, based on the positional relationship between the logistics object and the conveyor line operation area, detecting the logistics operation status of the conveyor line includes: determining the object transportation trajectory of the logistics object based on the corresponding positional relationship of the logistics objects in the multiple frames of conveyor line operation images; determining the trajectory difference between the object transportation trajectory and a preset transportation trajectory related to the conveyor line operation area; and generating an object drop detection result in response to the trajectory difference satisfying a preset difference condition, wherein the object drop detection result characterizes that a logistics object has dropped during the operation of the conveyor line.

[0012] According to an embodiment of the present disclosure, the conveyor line operation area includes N area boundaries, where N is an integer greater than 3; wherein, determining the trajectory difference between the object transport trajectory and the preset transport trajectory of the conveyor line includes: determining at least two target boundaries intersecting with the object transport trajectory from multiple area boundaries; and determining the target trajectory difference that satisfies the preset difference condition in response to the area boundary adjacent to the at least two target boundaries.

[0013] According to an embodiment of the present disclosure, it also includes: detecting the conveyor line operation image to obtain a transport object, the transport object is used to transport a logistics object, so that the logistics object can leave the conveyor line; determining the object area overlap between the transport object and the dropped object, the dropped object is a logistics object corresponding to the target trajectory difference, and the target trajectory difference satisfies a preset difference condition; wherein, in response to the trajectory difference satisfying the preset difference condition, generating an object drop detection result includes: in response to the trajectory difference satisfying the preset difference condition, and the object area overlap does not satisfy the preset overlap condition, generating an object drop detection result.

[0014] Another aspect of the present disclosure provides a logistics operation status detection device, including: an acquisition module, used to acquire a conveyor line operation image; a target detection module, used to perform target detection on the conveyor line operation image to obtain a logistics object transported by the conveyor line, the conveyor line operation image includes a designated conveyor line operation area; a first detection module, used to detect the logistics operation status of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area.

[0015] Another aspect of the present disclosure provides an electronic device, comprising:

[0016] one or more processors;

[0017] a memory for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0019] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0020] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0021] According to the embodiments of the present disclosure, by detecting the logistics objects being transported in the conveyor line operation image and detecting the logistics operation status of the conveyor line based on the positional relationship between the logistics objects and the conveyor line operation area, the transportation situation of the logistics objects in the designated conveyor line operation area can be more accurately characterized, thereby being able to more accurately and timely determine the logistics operation status of the conveyor line and improve logistics efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 The system architecture diagram to which the logistics operation status detection can be applied according to the embodiment of the present disclosure is schematically shown;

[0024] Figure 2 A flow chart of a method for detecting a logistics operation status according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 3 A schematic diagram schematically shows a conveyor line operation image according to an embodiment of the present disclosure;

[0026] Figure 4 A flowchart schematically shows a flow chart of generating an abnormality detection result representing an abnormality in a logistics operation state of a conveyor line in response to a logistics object staying for a time period greater than or equal to a first preset time period threshold according to an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of a card according to an embodiment of the present disclosure is schematically shown;

[0028] Fig. 6A A schematic diagram schematically shows a trajectory difference that does not meet a preset difference condition according to an embodiment of the present disclosure;

[0029] Figure 6B A schematic diagram schematically shows a target trajectory difference that satisfies a preset difference condition according to an embodiment of the present disclosure;

[0030] Figure 6C A schematic diagram of a dropped part according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 7 A flowchart of generating an object drop detection result in response to a trajectory difference satisfying a preset difference condition according to an embodiment of the present disclosure is schematically shown;

[0032] Figure 8 A block diagram schematically shows a logistics operation status detection device according to an embodiment of the present disclosure; and

[0033] Fig. 9 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0039] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0040] An embodiment of the present disclosure provides a method for detecting a logistics operation status, comprising: acquiring a conveyor line operation image; performing target detection on the conveyor line operation image to obtain a logistics object transported by the conveyor line, wherein the conveyor line operation image includes a designated conveyor line operation area; and detecting the logistics operation status of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area.

[0041] Figure 1 The following schematically shows a system architecture diagram that can be applied to logistics operation status detection according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0042] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0043] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0044] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0045] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0046] It should be noted that the logistics operation status detection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the logistics operation status detection device provided in the embodiment of the present disclosure can generally be set in the server 105. The logistics operation status detection method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the logistics operation status detection device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the logistics operation status detection method provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the logistics operation status detection device provided in the embodiment of the present disclosure can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0047] For example, the conveyor line operation image may be originally stored in any one of the first terminal device 101, the second terminal device 102 or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored on an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the logistics operation state detection method provided in the embodiment of the present disclosure, or send the conveyor line operation image to other terminal devices, servers, or server clusters, and the logistics operation state detection method provided in the embodiment of the present disclosure may be executed by the server 105. Accordingly, the logistics operation state detection device provided in the embodiment of the present disclosure may generally be arranged in the server 105. The logistics operation state detection method provided in the embodiment of the present disclosure may also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the logistics operation status detection device provided in the embodiment of the present disclosure may also be arranged in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the logistics operation status detection method provided in the embodiment of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or may also be executed by other terminal devices that are different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the logistics operation status detection device provided in the embodiment of the present disclosure may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or may be arranged in other terminal devices that are different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0048] For example, the conveyor line operation image may be originally stored in any one of the first terminal device 101, the second terminal device 102 or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the logistics operation status detection method provided by the embodiment of the present disclosure, or send the conveyor line operation image to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the conveyor line operation image may execute the logistics operation status detection method provided by the embodiment of the present disclosure.

[0049] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0050] Figure 2 The flowchart of the logistics operation status detection method according to the embodiment of the present disclosure is schematically shown.

[0051] like Figure 2 As shown, the method includes operations S210 to S230.

[0052] In operation S210 , a conveyor line operation image is acquired.

[0053] According to the embodiments of the present disclosure, the conveyor line may be a conveyor line with rollers and belts, or a chain conveyor line. The embodiments of the present disclosure do not limit the specific type or structure of the conveyor line, as long as it can convey items.

[0054] According to an embodiment of the present disclosure, the conveyor line operation image may be acquired by an image acquisition device installed above the conveyor line. For example, the image acquisition device may be a camera installed diagonally above the conveyor line, which may capture the transportation conditions of the conveyor line in real time.

[0055] In operation S220, target detection is performed on the conveyor line operation image to obtain the logistics objects transported by the conveyor line.

[0056] According to an embodiment of the present disclosure, the logistics object may be a courier package or a conveyed item.

[0057] According to an embodiment of the present disclosure, the target detection method may be a YOLO (You Only Look Once) algorithm, a neural network algorithm, etc. For example, the target detection algorithm may be the S model in YOLOV8 (YOLO Version8). It should be noted that the target detection method is not limited, and different target detection algorithms may be selected according to the complexity of the scene to identify the logistics objects transported by the conveyor line.

[0058] For example, the target detection of logistics objects can be determined by a target detection algorithm based on deep learning. The deep learning model requires a large amount of labeled data, so a semi-supervised method is used to label the data. First, some logistics objects are manually labeled using a labeling tool, and then the labeled data is used to detect logistics objects on unlabeled visual data. Then, the prediction results of the model are manually corrected, and the corrected data is used to continue training the model. This process is repeated several times until the model's detection accuracy for logistics objects reaches a certain threshold, so as to obtain a trained target detection algorithm based on deep learning.

[0059] According to an embodiment of the present disclosure, the conveyor line operation image includes a specified conveyor line operation area. For example, the specified conveyor line operation area can be obtained by performing target detection through a deep learning model. The specified conveyor line operation area can be a congested area in the conveyor line or a preset conveyor line operation area.

[0060] For example, the designated conveyor line operation area may be where the number of logistics objects exceeds the preset number and there is overlapping stacking between logistics objects, that is, the designated conveyor line operation area may be an area where congestion occurs. The designated conveyor line operation area may be determined by first performing target detection through the YOLO algorithm to obtain the logistics objects transported by the conveyor line, and then counting the number of logistics objects transported by the conveyor line in each conveyor line operation area or identifying whether there is overlapping stacking between logistics objects through a deep learning model.

[0061] The number of logistics objects in the conveyor line operation area can be calculated using the SORT (Simple Online and Realtime Tracking) algorithm, which has low computational complexity. The SORT algorithm is based on Kalman filtering and the Hungarian algorithm, has a fast running speed, and can achieve faster than real-time tracking without relying on GPU (Graphics Processing Unit) devices.

[0062] For example, the designated conveyor line operation area can be obtained by statistics of the logistics operation status of the historical conveyor line operation images, that is, the designated conveyor line operation area can be an area where abnormal logistics operation status is prone to occur. Object detection is performed on the conveyor line operation image to obtain the logistics objects transported by the conveyor line in the designated conveyor line operation area.

[0063] Figure 3 A schematic diagram of a conveyor line operation image according to an embodiment of the present disclosure is schematically shown.

[0064] like Figure 3 The conveyor line operation image includes logistics objects and designated conveyor line operation areas. The logistics objects may be items within target detection frames in purple, green, or blue, and the designated conveyor line operation areas may be areas within target detection frames in red.

[0065] The target detection boxes of the logistics object and the designated conveyor line operation area are all quadrilaterals composed of four points. The four vertices of each designated conveyor line operation area can be manually marked preset areas, or can be automatically obtained using a deep learning model, without specific restrictions.

[0066] In operation S230, the logistics operation state of the conveyor line is detected based on the positional relationship between the logistics object and the operation area of ​​the conveyor line.

[0067] According to an embodiment of the present disclosure, the positional relationship between the logistics object and the conveyor line operating area may include the logistics object at the edge of the conveyor line operating area, the logistics object within the conveyor line operating area, the logistics object outside the conveyor line operating area, etc.

[0068] For example, whether the center point of the logistics object is within the target detection frame of the conveyor line operation area can be judged by whether the center point is within the target detection frame. If the center point is within the target detection frame, the logistics object is within the conveyor line operation area, otherwise the logistics object is not within the conveyor line operation area. Specifically, assuming that the center point M is inside the target detection frame ABCD, the sum of the areas of the triangle formed by M and any two points of ABCD is equal to the area of ​​ABCD. If not inside, the sum of the areas is greater than the area of ​​ABCD.

[0069] According to an embodiment of the present disclosure, the logistics operation status may characterize the transmission status of logistics objects in the conveyor line.

[0070] For example, one or more logistics objects in the conveyor line operation image are at the edge of the conveyor line operation area or the logistics objects are outside the conveyor line operation area, which may cause an abnormality in the logistics operation status.

[0071] For example, if there are multiple logistics objects in the conveyor line operation area in the conveyor line operation image, the logistics operation status of the conveyor line may be normal.

[0072] According to the embodiments of the present disclosure, by detecting the logistics objects being transported in the conveyor line operation image and detecting the logistics operation status of the conveyor line based on the positional relationship between the logistics objects and the conveyor line operation area, the transportation situation of the logistics objects in the designated conveyor line operation area can be more accurately characterized, thereby enabling the logistics operation status of the conveyor line to be determined more accurately and timely, thereby improving logistics efficiency.

[0073] According to an embodiment of the present disclosure, based on the positional relationship between a logistics object and an operating area of ​​a conveyor line, detecting the logistics operating status of a conveyor line includes: determining the duration of a logistics object's stay in the operating area of ​​a conveyor line according to the positional relationship; and generating an abnormality detection result characterizing an abnormality in the logistics operating status of the conveyor line in response to a stay duration of the logistics object being greater than or equal to a first preset duration threshold.

[0074] According to the embodiments of the present disclosure, since the image acquisition device for acquiring the conveyor line operation image has an acquisition frequency, the time information can be determined according to the acquired multiple frames of the conveyor line operation image.

[0075] For example, the object stay time of the logistics object in the conveyor line operation area is determined based on the positional relationship between the logistics object and the conveyor line operation area in multiple frames of conveyor line operation images.

[0076] According to an embodiment of the present disclosure, the first preset time threshold may be a time threshold during which the logistics object does not leave the conveyor line operation area.

[0077] According to the embodiments of the present disclosure, since the image acquisition device for acquiring the conveyor line operation image has an acquisition frequency, the object residence time of the logistics object in the conveyor line operation area can be quickly determined according to the position relationship. It is determined whether the object residence time is greater than or equal to the first preset time threshold, thereby determining whether the conveyor line logistics operation state is abnormal, realizing rapid abnormality detection, and improving logistics efficiency.

[0078] According to an embodiment of the present disclosure, in response to a logistics object's residence time being greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line includes: in response to a logistics object's residence time being greater than or equal to a first preset time threshold, determining a size ratio relationship between an object size of a first logistics object and an area size of an operating area of ​​the conveyor line, the first logistics object being a logistics object located in the operating area of ​​the conveyor line; in response to the size ratio relationship satisfying a preset size ratio condition, generating an object congestion detection result, the abnormality detection result including an object congestion detection result.

[0079] According to the embodiments of the present disclosure, the abnormal state of the conveyor line logistics operation state may include blockage, jamming and dropped parts of the logistics objects. Blockage means that multiple logistics objects in the designated conveyor line operation area are congested, resulting in the inability of the following logistics objects to pass through the designated conveyor line operation area. Jam means that the following logistics objects can still pass through the designated conveyor line operation area, but some logistics objects do not leave the designated conveyor line operation area for a long time. Dropped parts means that the logistics objects fall from the conveyor line in the designated conveyor line operation area to outside the conveyor line.

[0080] According to the embodiments of the present disclosure, under normal circumstances, logistics objects continue to move along the conveyor line, and within a certain period of time, there will be a conveyor line operation area where logistics objects are densely distributed. When a blockage or jam occurs, the logistics object will stay in the conveyor line operation area for a time that exceeds the conventional first preset time threshold n.

[0081] According to an embodiment of the present disclosure, the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area may be the size ratio of the width of the first logistics object in the same direction to the width of the conveyor line.

[0082] According to an embodiment of the present disclosure, the preset size ratio condition may be greater than or equal to the preset size ratio, for example, the preset size ratio may be 75%. There is no specific restriction on the preset size ratio, and it may be set according to a specific scenario.

[0083] According to the embodiment of the present disclosure, when it is determined that the retention time of the logistics object in the conveyor line operation area exceeds the conventional first preset time threshold n, it can be concluded that the conveyor line logistics operation state is abnormal. Then, it is determined whether the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area meets the preset size ratio condition, thereby determining the abnormality type, for example, if the size ratio relationship meets the preset size ratio condition, an object congestion detection result is generated.

[0084] According to the embodiments of the present disclosure, when congestion of logistics objects occurs on the conveyor line, the conveyor line is paralyzed, resulting in untimely delivery and affecting user experience; at the same time, the congested logistics objects may be squeezed and damaged, resulting in increased compensation costs.

[0085] Therefore, by responding to the logistics object's residence time being greater than or equal to the first preset time threshold, the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area is promptly determined; and then in response to the size ratio relationship satisfying the preset size ratio condition, the object congestion detection result is quickly generated to avoid congestion on the conveyor line, thereby improving the delivery efficiency of subsequent logistics objects while ensuring the quality of the logistics objects and enhancing the user experience.

[0086] According to an embodiment of the present disclosure, in response to a logistics object staying time greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line includes: in response to a logistics object staying time greater than or equal to a second preset time threshold, generating an object congestion detection result, wherein the time represented by the first preset time threshold is greater than the time represented by the second preset time threshold.

[0087] According to an embodiment of the present disclosure, the first preset duration threshold n and the second preset duration threshold m

[0088] There can be two hyperparameters, which can be formulated based on experience. Specifically, n can be determined based on the size of the conveyor line operation area and the flow rate. The larger the conveyor line operation area, the larger n. Some conveyor lines stop intermittently, and the flow rate is slow, so n should also be larger. n is generally the time to complete the conveyor line operation area plus a value. m is generally less than n, and m is generally the time to complete the conveyor line operation area.

[0089] According to an embodiment of the present disclosure, if the logistics object cannot be transported out of the conveyor line operation area within the second preset time threshold, congestion will occur in the conveyor line operation area.

[0090] According to the embodiments of the present disclosure, by determining whether the residence time of a logistics object is greater than or equal to a second preset time threshold, it is possible to quickly detect whether congestion occurs in the conveyor line operation area, thereby improving logistics efficiency.

[0091] According to an embodiment of the present disclosure, in response to a logistics object's residence time being greater than or equal to a first preset time threshold, generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line also includes: in response to at least one logistics object's residence time being greater than or equal to the first preset time threshold, the object residence time of all logistics objects in the conveyor line operation area being greater than a second preset time threshold, and the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not satisfy a preset size ratio condition, generating an object card detection result, wherein the abnormality detection result includes an object card detection result, and the first logistics object is a logistics object staying in the conveyor line operation area.

[0092] According to the embodiments of the present disclosure, since there are multiple logistics objects in the conveyor line, it can be first determined whether there is a logistics object whose residence time is greater than or equal to the first preset time threshold among the multiple logistics objects. If so, there is an abnormality in the logistics operation state of the conveyor line. In the case of an abnormality in the logistics operation state of the conveyor line, the object residence time of all logistics objects in the conveyor line operation area is greater than the second preset time threshold, and the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not meet the preset size ratio condition, then a serious jamming situation occurs on the conveyor line, and the object jamming detection result is generated in time, and an alarm message is immediately issued to quickly clear the conveyor line.

[0093] According to the embodiments of the present disclosure, by judging whether the residence time of at least one logistics object is greater than or equal to a first preset time threshold, whether the residence time of all logistics objects in the conveyor line operation area is greater than a second preset time threshold, and whether the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not meet the preset size ratio condition, it is possible to simply and quickly judge whether a jam occurs on the conveyor line, thereby improving detection efficiency.

[0094] Figure 4 A flowchart is schematically shown for generating an abnormality detection result characterizing an abnormality in the logistics operation state of a conveyor line in response to a logistics object residence time being greater than or equal to a first preset time threshold according to an embodiment of the present disclosure.

[0095] In operation S410, the duration of a logistics object's stay in a conveyor line operation area is determined based on the position relationship.

[0096] In operation S420, whether the object stay time is greater than or equal to the first preset time threshold, if so, perform operation S430; if not, perform operation S460.

[0097] In operation S430, a size ratio relationship between an object size of the first logistics object and an area size of the conveyor line operation area is determined.

[0098] In operation S440, it is determined whether the size ratio relationship satisfies a preset size ratio condition. If so, operation S450 is performed, and if not, operation S470 is performed.

[0099] In operation S450, an object congestion detection result is generated.

[0100] In operation S460, it is determined whether the object stay time is greater than or equal to a second preset time threshold, if so, operation S450 is executed, if not, operation S470 is executed.

[0101] According to an example of the present disclosure, when the object stay time is greater than or equal to the first preset time threshold and the size ratio relationship meets the preset size ratio condition, an object congestion detection result is generated. Or when the object stay time is not greater than or equal to the first preset time threshold, but the object stay time is greater than or equal to the second preset time threshold, an object congestion detection result is generated. The object congestion result can be parameter 1.

[0102] In operation S470, it is determined that the size ratio relationship does not satisfy a preset size ratio condition.

[0103] In operation S480, an object card detection result is generated.

[0104] According to an embodiment of the present disclosure, when the object stays for a duration greater than or equal to a second preset duration threshold and the size ratio relationship does not satisfy a preset size ratio condition, an object card detection result is generated.

[0105] Figure 5 A schematic diagram of a card according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 5 As shown in the figure, the yellow target detection box marks the logistics object of the stuck owner.

[0107] According to an embodiment of the present disclosure, based on the positional relationship between the logistics objects and the conveyor line operating area, detecting the logistics operating status of the conveyor line also includes: determining a congestion index parameter characterizing the object congestion situation of the conveyor line according to a size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operating area, wherein the first logistics object is a logistics object located in the conveyor line operating area; and sending the congestion index parameter to a designated client.

[0108] According to the embodiments of the present disclosure, the congestion index parameter characterizing the object congestion of the conveyor line can be determined based on the dimensional ratio relationship between the width of the first logistics object in the same direction and the width of the conveyor line operation area, and the dimensional ratio relationship between the length of the first logistics object in the same direction and the length of the conveyor line operation area. For example, the ratio of the area of ​​the first logistics object to the area of ​​the entire conveyor line operation area is calculated to determine the congestion index parameter characterizing the object congestion of the conveyor line.

[0109] For example, the first logistics object includes a and b, the width ratio is 0.4, 03, the length ratio is 0.2, 05, and there is no overlap between a and b. Then the area ratio of a can be 0.08, the area ratio of b can be 0.15, and the area ratio of the first logistics object to the entire conveyor line operation area can be 0.08 + 0.15 = 0.23. If a and b have overlap, the area of ​​the overlap needs to be removed.

[0110] According to the embodiments of the present disclosure, the ratio of the area of ​​the first logistics object to the area of ​​the entire conveyor line operation area can be used as a congestion index parameter to adjust the loading rate. In addition, it is necessary to update according to whether a blockage occurs. After congestion occurs, the area ratio may not reach 1. At this time, it is necessary to quickly set the congestion index parameter to 1. That is, the congestion index parameter can include two results. For example, when there is no congestion in the conveyor line operation area, it is the area ratio. If congestion occurs in the conveyor line operation area, the congestion index parameter is set to 1 within 1 second.

[0111] According to an embodiment of the present disclosure, the designated client may be a monitoring terminal of a logistics status maintainer, and the logistics status maintainer may timely maintain the normal transportation of the conveyor line.

[0112] According to an embodiment of the present disclosure, a congestion index parameter characterizing the object congestion condition of the conveyor line can be directly determined based on the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area; the congestion index parameter is sent to a designated client to facilitate timely maintenance of normal transportation of the conveyor line and improve logistics efficiency.

[0113] According to an embodiment of the present disclosure, a conveyor line operation image includes multiple frames; wherein, based on the positional relationship between the logistics object and the conveyor line operation area, detecting the logistics operation status of the conveyor line includes: determining the object transportation trajectory of the logistics object based on the corresponding positional relationship of the logistics objects in the multiple frames of conveyor line operation images; determining the trajectory difference between the object transportation trajectory and a preset transportation trajectory related to the conveyor line operation area; and generating an object drop detection result in response to the trajectory difference satisfying a preset difference condition, wherein the object drop detection result characterizes that a logistics object has dropped during the operation of the conveyor line.

[0114] According to an embodiment of the present disclosure, multiple frames may include conveyor line operation images corresponding to the starting point of a logistics object entering a conveyor line operation area, the end point of a logistics object exiting a conveyor line operation area, and a preset point of a logistics object in a preset area of ​​the conveyor line operation area. Among them, the preset point of a logistics object in a preset area of ​​the conveyor line operation area may be any point in any area of ​​the conveyor line operation area. Multiple frames also have time information, and the corresponding positional relationships of logistics objects in multiple frames of conveyor line operation images can be connected to obtain the object transportation trajectory of the logistics object, and the positional relationship of the logistics object at each time point can be intuitively obtained.

[0115] According to an embodiment of the present disclosure, the corresponding positional relationship of the logistics objects in multiple frames of conveyor line operation images can be used to determine the object transportation trajectory of the logistics objects in the transfer point operation area.

[0116] According to an embodiment of the present disclosure, the preset transportation trajectory can represent that the logistics objects are in the operation area of ​​the transfer point during the transmission process.

[0117] According to an embodiment of the present disclosure, the trajectory difference satisfies a preset difference condition, that is, it represents the situation that the logistics object is not within the operating area of ​​the transmission point during the transmission process, and generates an object drop detection result representing that the logistics object falls during the operation of the conveyor line.

[0118] According to the embodiments of the present disclosure, based on the positional relationship of the logistics objects in the multiple frames of conveyor line operation images, the object transportation trajectory of the logistics objects within the multiple frames is accurately determined; the trajectory difference between the object transportation trajectory and the preset transportation trajectory related to the conveyor line operation area is determined. Then, in response to the trajectory difference satisfying the preset difference condition, the object drop detection result is generated to improve the accuracy of the drop detection.

[0119] According to an embodiment of the present disclosure, the conveyor line operation area includes N area boundaries, where N is an integer greater than 3; wherein, determining the trajectory difference between the object transport trajectory and the preset transport trajectory of the conveyor line includes: determining at least two target boundaries intersecting with the object transport trajectory from multiple area boundaries; and determining the target trajectory difference that satisfies the preset difference condition in response to the area boundary adjacent to the at least two target boundaries.

[0120] According to an embodiment of the present disclosure, the N area boundaries may be of any linear shape, such as a straight line, a curve, etc. For example, the conveyor line operation area includes four area boundaries, and all of them are straight lines, such as Figure 2 The target detection frame of the conveyor line operation area in .

[0121] Fig. 6A A schematic diagram schematically shows a trajectory difference that does not meet a preset difference condition according to an embodiment of the present disclosure.

[0122] like Fig. 6A As shown, the conveyor line operation area in the figure can be the area within the red frame, and the four sides of the red frame can be the four area boundaries.

[0123] The line segment formed by connecting the center point A before the logistics object enters the conveyor line operation area (the area in the red box) and the center point B after the logistics object leaves the conveyor line operation area, the line segment AB can be the object transportation trajectory. The line segment AB and the two edges of the conveyor line operation area have an intersection, that is, at least two target boundaries that intersect with the object transportation trajectory. Fig. 6A The two target boundaries are the two opposite sides of the red box, which means normal transmission (no parts dropped).

[0124] Figure 6B A schematic diagram schematically shows a target trajectory difference that meets a preset difference condition according to an embodiment of the present disclosure.

[0125] like Figure 6B As shown in the figure, the conveyor line operation area can be the area in the red frame, and the four sides of the red frame can be the four area boundaries. The line segment CD formed by the center point C before the logistics object enters the conveyor line operation area (the area in the red frame) and the center point D after the logistics object leaves the conveyor line operation area can be the object transportation trajectory. The line segment CD and the two sides of the conveyor line operation area have an intersection, that is, at least two target boundaries intersecting with the object transportation trajectory. Figure 6B If the two target boundaries are the adjacent edges of the red box and the red box respectively, the logistics object is a dropped item. According to the embodiment of the present disclosure, the above-mentioned dropped item situation can be converted into a geometric problem of whether there is an intersection between CD and the vertical line of the red box (that is, the boundary of the area parallel to the transportation direction of the conveyor line). To determine whether the two points that make up a line segment are on the same side of another line segment, it can be achieved by calculating the sign of the cross product of the vectors formed by these two points and the two end points of the line segment. If the signs of the two cross products are the same, the two points are on the same side of the line segment; if the signs are different, they are on both sides of the line segment. If one or both of the cross products are zero, it means that at least one point is on the straight line where the line segment is located. If the signs are different, they are on both sides of the line segment, then there is an intersection, and a dropped item has occurred.

[0126] Figure 6C A schematic diagram of a dropped part according to an embodiment of the present disclosure is schematically shown.

[0127] like Figure 6C As shown in the figure, line segment C'D' can be the object transportation trajectory. C' can be the center point of the logistics object before entering the conveyor line operation area (the area in the red box), and D' can be the center point of the logistics object after leaving the conveyor line operation area. If line segment C'D' intersects with the two adjacent area boundaries of the conveyor line operation area, the logistics object falls outside the conveyor line, that is, the logistics object is a dropped part.

[0128] According to an embodiment of the present disclosure, at least two target boundaries intersecting with the object transportation trajectory are determined from multiple area boundaries; and then the target trajectory difference that meets the preset difference condition is directly determined in response to the area boundary adjacent to the at least two target boundaries, so that the dropped parts detection can be quickly realized.

[0129] According to an embodiment of the present disclosure, it also includes: detecting the conveyor line operation image to obtain a transport object, the transport object is used to transport a logistics object, so that the logistics object can leave the conveyor line; determining the object area overlap between the transport object and the dropped object, the dropped object is a logistics object corresponding to the target trajectory difference, and the target trajectory difference satisfies a preset difference condition; wherein, in response to the trajectory difference satisfying the preset difference condition, generating an object drop detection result includes: in response to the trajectory difference satisfying the preset difference condition, and the object area overlap does not satisfy the preset overlap condition, generating an object drop detection result.

[0130] According to the embodiments of the present disclosure, considering that the interference of the transport object is excluded in the dropped item, sometimes the transport object takes away the logistics object on the conveyor line. If the dropped item detection method is used, the transport object may be predicted as a dropped item. Therefore, it is necessary to avoid such false alarms.

[0131] According to an embodiment of the present disclosure, the transport object may be a robotic arm, a human hand, or the like.

[0132] According to an embodiment of the present disclosure, the object area overlap between the transported object and the dropped object may be obtained by inputting the conveyor line image into a machine learning algorithm.

[0133] For example, the object being carried can be a human hand. Images with labeled people and human bodies are selected as training data sets. The training set has 18,877 images and the test set has 1,629 images, containing 52,050 and 5,983 hand instances respectively. The experimental results show that the trained model can detect human hands very well. Specifically, the data set classifies the attributes of hands into 4 types: hands holding objects, empty, hands in contact with oneself, and hands in contact with others. Change the 4 categories to 1 category, namely, hands. Then calculate the rectangle from the labeled quadrilateral and use it as the true value for YOLO detection for training.

[0134] If in one quarter of the video frames, the overlap between the object area of ​​the dropped object and the object area of ​​the human hand exceeds the preset overlap condition, it is considered that a person has taken the dropped object instead of dropping the item.

[0135] Compared with the method of using pressure sensors and photoelectric sensors, using the existing surveillance cameras in the park can improve the comprehensive utilization rate of surveillance cameras. The camera is required to be able to capture the conveyor line, without additional hardware costs and installation fees. At the same time, the intelligent algorithm based on computer vision has strong robustness, strong generalization ability, and low false alarm rate.

[0136] According to the embodiments of the present disclosure, the detection of three situations, namely, congestion, stuck parts, and dropped parts, can be realized, and the congestion index coefficient can be predicted, and the congestion can be warned according to the size of the congestion index coefficient. In addition, the interference caused by the transported objects is also considered, and the robustness is stronger. In short, the embodiments of the present disclosure reduce hardware costs, enhance detection robustness, and improve detection accuracy.

[0137] Figure 7 The flowchart for generating an object drop detection result in response to a trajectory difference satisfying a preset difference condition according to an embodiment of the present disclosure is schematically shown.

[0138] In operation S710, an object transportation trajectory of the logistics object is determined based on the corresponding position relationship of the logistics objects in the multiple frames of conveyor line operation images.

[0139] In operation S720, at least two target boundaries intersecting the object transportation track are determined from among the plurality of area boundaries.

[0140] In operation S730 , in response to a region boundary adjacent to at least two target boundaries, a target trajectory difference that satisfies a preset difference condition is determined.

[0141] In operation S740, the conveyor line operation image is detected to obtain a transport object, which is used to transport a logistics object so that the logistics object can leave the conveyor line. The object area overlap between the transport object and the dropped object is determined. The dropped object is a logistics object corresponding to the target trajectory difference, and the target trajectory difference satisfies a preset difference condition.

[0142] In operation S750, in response to the trajectory difference satisfying a preset difference condition and the object region coincidence not satisfying a preset coincidence condition, an object drop detection result is generated.

[0143] Figure 8 A block diagram of a logistics operation status detection device according to an embodiment of the present disclosure is schematically shown.

[0144] like Figure 8 As shown, the logistics operation status detection device 800 includes an acquisition module 810, a target detection module 820 and a first detection module 830.

[0145] The acquisition module 810 is used to acquire the conveyor line operation image. In one embodiment, the acquisition module 810 can be used to perform the operation S210 described above, which will not be described in detail here.

[0146] The target detection module 820 is used to perform target detection on the conveyor line operation image to obtain the logistics object transported by the conveyor line. In one embodiment, the target detection module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0147] The first detection module 830 is used to detect the logistics operation state of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area. In one embodiment, the first detection module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0148] According to an embodiment of the present disclosure, the first detection module 830 includes a first determination submodule and a first generation submodule. The first determination submodule is used to determine the object residence time of the logistics object in the conveyor line operation area according to the position relationship. The generation submodule is used to generate an abnormality detection result characterizing the abnormality of the conveyor line logistics operation state in response to the logistics object residence time being greater than or equal to the first preset time threshold.

[0149] According to an embodiment of the present disclosure, the first generation submodule includes a first determination unit and a first generation unit. The first determination unit is used to determine the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area in response to the length of stay of the logistics object being greater than or equal to the first preset time threshold, and the first logistics object is a logistics object located in the conveyor line operation area. The first generation unit is used to generate an object congestion detection result in response to the size ratio relationship satisfying a preset size ratio condition, and the abnormal detection result includes the object congestion detection result.

[0150] According to an embodiment of the present disclosure, the first generation submodule includes a second generation unit. The second generation unit generates an object congestion detection result in response to the logistics object staying for a time greater than or equal to a second preset time threshold, wherein the time represented by the first preset time threshold is greater than the time represented by the second preset time threshold.

[0151] According to an embodiment of the present disclosure, the first generation submodule further includes a third generation unit. The third generation unit is used to generate an object card detection result in response to at least one logistics object staying for a time greater than or equal to a first preset time threshold, the object stay time of all logistics objects in the conveyor line operation area is greater than a second preset time threshold, and the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not meet the preset size ratio condition, wherein the abnormal detection result includes the object card detection result, and the first logistics object is a logistics object staying in the conveyor line operation area.

[0152] According to an embodiment of the present disclosure, the first detection module 830 further includes a second determination submodule and a sending submodule. The second determination submodule is used to determine a congestion index parameter characterizing the object congestion of the conveyor line according to the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area, wherein the first logistics object is a logistics object located in the conveyor line operation area. The sending submodule is used to send the congestion index parameter to a specified client.

[0153] According to an embodiment of the present disclosure, the conveyor line operation image includes multiple frames; wherein the first detection module 830 includes a third determination submodule, a fourth determination submodule and a second generation submodule. The third determination submodule is used to determine the object transportation trajectory of the logistics object based on the corresponding positional relationship of the logistics objects in the multiple frames of the conveyor line operation image. The fourth determination submodule is used to determine the trajectory difference between the object transportation trajectory and the preset transportation trajectory related to the conveyor line operation area. The second generation submodule is used to generate an object drop detection result in response to the trajectory difference satisfying the preset difference condition, wherein the object drop detection result characterizes that the conveyor line generates a logistics object drop during the operation process.

[0154] According to an embodiment of the present disclosure, the conveyor line operation area includes N area boundaries, where N is an integer greater than 3; wherein the fourth determination submodule includes a second determination unit and a third determination unit. The second determination unit is used to determine at least two target boundaries intersecting with the object transportation trajectory from the multiple area boundaries. The third determination unit is used to respond to the area boundary adjacent to the at least two target boundaries and determine the target trajectory difference that meets the preset difference condition.

[0155] According to an embodiment of the present disclosure, the above-mentioned device also includes a second detection module and a determination module. The second detection module is used to detect the conveyor line operation image to obtain a transport object, and the transport object is used to transport a logistics object so that the logistics object can leave the conveyor line. The determination module is used to determine the object area overlap between the transport object and the dropped object, and the dropped object is a logistics object corresponding to the target trajectory difference, and the target trajectory difference satisfies a preset difference condition; wherein the second generation submodule includes a fourth generation unit. The fourth generation unit is used to generate an object drop detection result in response to the trajectory difference satisfying the preset difference condition and the object area overlap not satisfying the preset overlap condition.

[0156] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0157] For example, any of the acquisition module 810, the target detection module 820, and the first detection module 830 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the target detection module 820, and the first detection module 830 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the acquisition module 810, the target detection module 820, and the first detection module 830 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0158] It should be noted that the logistics operation status detection device in the embodiment of the present disclosure corresponds to the logistics operation status detection method in the embodiment of the present disclosure. The description of the logistics operation status detection device part specifically refers to the logistics operation status detection method part, which will not be repeated here.

[0159] Fig. 9A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0160] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0161] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0162] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0163] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0164] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0165] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0166] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0167] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains a program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the logistics operation status detection method provided by the embodiment of the present disclosure.

[0168] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0169] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0170] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0172] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for detecting a logistics operation status, comprising: Acquire conveyor line operation images; Performing target detection on the conveyor line operation image to obtain a logistics object transported by the conveyor line, wherein the conveyor line operation image includes a designated conveyor line operation area; Based on the positional relationship between the logistics object and the conveyor line operation area, the logistics operation status of the conveyor line is detected.

2. The method according to claim 1, wherein: The detecting of the logistics operation state of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area comprises: Determine, based on the positional relationship, the object stay time of the logistics object in the conveyor line operation area; In response to the residence time of the logistics object being greater than or equal to a first preset time threshold, an abnormality detection result characterizing an abnormality in the logistics operation state of the conveying line is generated.

3. The method according to claim 2, wherein: In response to the logistics object staying time being greater than or equal to the first preset time threshold, generating an abnormality detection result indicating that the logistics operation state of the conveying line is abnormal includes: In response to the residence time of the logistics object being greater than or equal to a first preset time threshold, determining a size ratio relationship between an object size of a first logistics object and an area size of the conveyor line operation area, the first logistics object being a logistics object located in the conveyor line operation area; In response to the size ratio relationship satisfying a preset size ratio condition, an object congestion detection result is generated, and the abnormality detection result includes the object congestion detection result.

4. The method according to claim 2, wherein: In response to the logistics object staying time being greater than or equal to the first preset time threshold, generating an abnormality detection result indicating that the logistics operation state of the conveying line is abnormal includes: In response to the logistics object's stay time being greater than or equal to a second preset time threshold, an object congestion detection result is generated, wherein the time represented by the first preset time threshold is greater than the time represented by the second preset time threshold.

5. The method according to any one of claims 2 to 4, wherein: In response to the logistics object staying for a time period greater than or equal to a first preset time period threshold, generating an abnormality detection result indicating that the logistics operation state of the conveying line is abnormal also includes: In response to at least one of the logistics objects having a residence time greater than or equal to a first preset time threshold, the object residence time of all logistics objects in the conveyor line operation area being greater than a second preset time threshold, and the size ratio relationship between the object size of the first logistics object and the area size of the conveyor line operation area does not meet the preset size ratio condition, an object card detection result is generated, wherein the abnormal detection result includes the object card detection result, and the first logistics object is a logistics object staying in the conveyor line operation area.

6. The method according to claim 1, wherein: The detecting the logistics operation state of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area further comprises: Determining a congestion index parameter characterizing the object congestion of the conveyor line according to a size ratio relationship between an object size of a first logistics object and an area size of the conveyor line operation area, wherein the first logistics object is a logistics object located in the conveyor line operation area; and The congestion indicator parameter is sent to a designated client.

7. The method according to claim 1, wherein: The conveyor line operation image includes multiple frames; Wherein, based on the positional relationship between the logistics object and the conveyor line operation area, detecting the logistics operation state of the conveyor line includes: Determining the object transportation trajectory of the logistics object based on the corresponding position relationship of the logistics objects in the multiple frames of the conveyor line operation images; Determining a trajectory difference between the object transport trajectory and a preset transport trajectory associated with the conveyor line operation area; and In response to the trajectory difference satisfying a preset difference condition, an object drop detection result is generated, wherein the object drop detection result indicates that a logistics object has dropped during the operation of the conveyor line.

8. The method according to claim 7, wherein: The conveyor line operation area includes N area boundaries, where N is an integer greater than 3; Wherein, determining the trajectory difference between the object transportation trajectory and the preset transportation trajectory of the conveyor line includes: determining at least two target boundaries intersecting the object transport trajectory from the plurality of area boundaries; In response to a region boundary adjacent to the at least two target boundaries, a target trajectory difference satisfying the preset difference condition is determined.

9. The method according to claim 7 or 8, further comprising: Detecting the conveyor line operation image to obtain a transport object, where the transport object is used to transport the logistics object so that the logistics object leaves the conveyor line; Determining an object area overlap between the transported object and the dropped object, the dropped object being a logistics object corresponding to a target trajectory difference, the target trajectory difference satisfying the preset difference condition; Wherein, in response to the trajectory difference satisfying a preset difference condition, generating an object drop detection result includes: In response to the trajectory difference satisfying a preset difference condition and the object area coincidence not satisfying a preset coincidence condition, the object drop detection result is generated.

10. A logistics operation status detection device, comprising: An acquisition module, used for acquiring the conveyor line operation image; A target detection module, used for performing target detection on the conveyor line operation image to obtain a logistics object transported by the conveyor line, wherein the conveyor line operation image includes a specified conveyor line operation area; The first detection module is used to detect the logistics operation status of the conveyor line based on the positional relationship between the logistics object and the conveyor line operation area.

11. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images