Transmission state detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a transmission state detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target video containing at least one to-be-detected transmission line; based on the target video, determining an article transmission flow corresponding to each to-be-detected conveying line at the target moment; wherein the article conveying flow is the number of articles passing through the to-be-detected conveying line in unit time; and for each to-be-detected conveying line, the conveying state of the to-be-detected conveying line at the target moment is determined based on the article conveying flow corresponding to the to-be-detected conveying line. According to the technical scheme, detection personnel do not need to manually determine the article conveying condition on the to-be-detected conveying line, the conveying state of the to-be-detected conveying line can be visually and effectively reflected through the article conveying flow corresponding to each to-be-detected conveying line, the article conveying condition of a logistics park can be comprehensively known, and the detection efficiency is improved. The effects of saving human resources and providing convenience for resource scheduling of the logistics park are achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to a transmission status detection method, device, electronic device and storage medium. Background Art

[0002] In the logistics park, items are transported through conveyor lines. In order to ensure that the conveyor lines can efficiently transport items to the destination and enable the normal operation of the logistics park, it is necessary to detect the transmission status of the items transported by each conveyor line in the logistics park.

[0003] In the prior art, inspection personnel are mainly arranged to inspect the transmission status of each conveyor line, so as to timely adjust the movement of goods based on the inspection results. However, in the process of implementing the present invention, it is found that the prior art has at least the following technical problems: in order to detect the transmission status of each conveyor line in the logistics park, a large number of inspection personnel need to be arranged to conduct inspections, which wastes precious human resources; and it is not convenient to fully understand the transportation status of goods in the logistics park, and it is impossible to effectively dispatch resources, which reduces the operation efficiency of the logistics park. Summary of the invention

[0004] The embodiments of the present invention provide a transmission status detection method, device, electronic device and storage medium to achieve the purpose of saving human resources and facilitating resource scheduling in a logistics park.

[0005] According to one aspect of the present invention, there is provided a transmission status detection method, comprising:

[0006] Acquire a target video containing at least one conveyor line to be detected;

[0007] Based on the target video, determine the object transmission flow rate corresponding to each of the to-be-detected conveyor lines at the target time; wherein the object transmission flow rate is the number of objects passing through the to-be-detected conveyor line per unit time;

[0008] For each of the conveyor lines to be detected, the transmission state of the conveyor line to be detected at the target time is determined based on the article transmission flow corresponding to the conveyor line to be detected.

[0009] According to another aspect of the present invention, there is provided a transmission status detection device, the device comprising:

[0010] A target video acquisition module, used to acquire a target video containing at least one conveyor line to be detected;

[0011] An item transmission flow determination module is used to determine the item transmission flow corresponding to each of the to-be-detected conveyor lines at a target time based on the target video; wherein the item transmission flow is the number of items passing through the to-be-detected conveyor line per unit time;

[0012] The transmission status determination module is used to determine the transmission status of each of the conveyor lines to be detected at the target time based on the article transmission flow corresponding to the conveyor line to be detected.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the transmission status detection method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transmission status detection method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention determines the item transmission flow rate corresponding to each conveyor line to be detected at the target time by acquiring a target video containing at least one conveyor line to be detected, wherein the item transmission flow rate is the number of items passing through the conveyor line to be detected per unit time; there is no need for detection personnel to manually determine the item transmission situation on the conveyor line to be detected; and, through the item transmission flow rate corresponding to each conveyor line to be detected, the transmission status of the conveyor line to be detected can be intuitively and effectively reflected, which is convenient for a comprehensive understanding of the item transportation situation in the logistics park, saves human resources, and provides convenience for resource scheduling in the logistics park.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a transmission status detection method provided according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of another transmission status detection method provided according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a transmission status detection process provided according to an embodiment of the present invention;

[0024] Figure 4 is a structural schematic diagram of a transmission status detection device provided according to an embodiment of the present invention;

[0025] Figure 5 It is a structural schematic diagram of an electronic device for implementing the transmission status detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "etc." and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] Before introducing the technical solution, an example of the application scenario can be first described. This technical solution can be applied in a logistics park, where the transmission status of each conveyor line when conveying items is detected by collecting target videos containing conveyor lines. Among them, the conveyor line may include a roller conveyor line, a roller conveyor line, a flat belt conveyor line, a chain conveyor line, and a bucket conveyor line. The items transmitted on the conveyor line may include the items themselves, or they may be packages, plastic boxes, cartons, and other containers after the items are packaged.

[0030] For example, the surveillance video collected by the monitoring equipment in the logistics park can be obtained and used as the target video for detection to determine the transmission status of each conveyor line when transporting items. Thus, without arranging manpower for detection, the transmission status of the conveyor line can be effectively determined, saving a lot of human resources and facilitating resource scheduling in the logistics park.

[0031] Figure 1 This is a flow chart of a transmission status detection method provided according to an embodiment of the present invention. This embodiment is applicable to the case where the transmission status of a conveyor line is detected when transmitting an article. The method can be executed by a transmission status detection device, which can be implemented in the form of hardware and / or software.

[0032] like Figure 1 As shown, the method of this embodiment may specifically include:

[0033] S110, acquiring a target video containing at least one conveyor line to be detected.

[0034] The conveyor line to be detected is a conveyor line used to transport goods in a logistics park, and the target video can be a video collected by an image acquisition device. In order to reduce the cost of detecting the transmission status, the existing monitoring equipment in the logistics park can be used as an image collector, and the monitoring video collected by the monitoring equipment can be used as the target video, so there is no need to install new image acquisition equipment, and there is no additional hardware cost and installation fee.

[0035] In the specific implementation, the target video can be obtained in real time or periodically; when the total number of items to be transmitted is greater than the preset item threshold, it means that the number of items to be transmitted is large, and the target video can be obtained in real time to timely detect the transmission status of the conveyor line to be detected, which is conducive to fast and effective resource scheduling in the logistics park and avoids the situation where the items are not transported in time. When the total number of items to be transmitted is less than or equal to the preset item threshold, it means that the number of items to be transmitted is small, and the target video can be obtained periodically to reduce the number of detections and reduce the workload.

[0036] S120: Based on the target video, determine the object transmission flow rate corresponding to each conveyor line to be inspected at the target time.

[0037] Among them, the item transmission flow rate is the number of items passing through the conveyor line to be detected per unit time; for example, the unit time can be 5 seconds. The target time is the time when the transmission state of the conveyor line to be detected is detected. Those skilled in the art can determine the target time according to the actual application situation. It should be noted that the detection area can be determined in advance for each conveyor line to be detected, and the item transmission flow rate in the detection area at the target time is determined as the item transmission flow rate corresponding to the conveyor line to be detected. Exemplarily, the detection area can be the contour area corresponding to the conveyor line to be detected, or it can be an area pre-set by the management personnel.

[0038] In a specific implementation, the target time can be determined as the end time or the start time, and the number of items passing through each conveyor line to be detected within the time period per unit time from the target time can be determined, and the number can be used as the item transmission flow corresponding to the conveyor line to be detected. For example, when the target time is the end time, the unit time is 5 seconds, and the number of items passing through each conveyor line to be detected within 5 seconds before the target time is 10, then the item transmission flow of the conveyor line to be detected is 10; if the target time is the start time, the number of items passing through each conveyor line to be detected within 5 seconds after the target time can be determined, and it can be used as the item transmission flow.

[0039] Alternatively, the number of items passing through each conveyor line to be detected can be determined with the target time as the starting time or the ending time and within the time period of the preset time interval from the target time; the preset time interval is determined as the time multiple of the unit time, and the number of items passing through each conveyor line to be detected is divided by the time multiple, and the obtained value is used as the item transmission flow of the conveyor line to be detected. For example, if the unit time is 5 seconds and the preset time interval is 10 seconds, the time multiple is 2. If the number of items passing through the conveyor line to be detected within the time period of the preset time interval from the starting time is 10, the item transmission flow of the conveyor line to be detected is 5.

[0040] S130. For each conveyor line to be detected, based on the article transmission flow rate corresponding to the conveyor line to be detected, determine the transmission state of the conveyor line to be detected at the target time.

[0041] The transmission state may include at least one of a congestion state, a normal working state and an idle state. The congestion state indicates that the items transmitted on the conveyor line to be detected at the target time are congested; the idle state indicates that the number of items on the conveyor line to be detected at the target time is less than the preset minimum number of items.

[0042] In this embodiment, the item transmission flow is related to the transmission state of the conveyor line to be detected. Specifically, if the item transmission flow is greater than or equal to the preset threshold, it can reflect that the conveyor line to be detected is in a normal working state at the target time. If the item transmission flow is less than the preset threshold, it can reflect that when the items on the conveyor line to be detected are congested and squeezed, it makes it difficult for the items to pass through the conveyor line to be detected, and the conveyor line to be detected is in a congested state at the target time; or, it reflects that the number of items on the conveyor line to be detected is small, and the conveyor line to be detected is in an idle state at the target time. It should be noted that those skilled in the art can determine the preset threshold according to the actual application situation. For example, the preset threshold can be set to 1.

[0043] When the item transmission flow is less than a preset threshold, in order to accurately determine the transmission status of the conveyor line to be detected at the target time, a prompt message can be generated and sent to the management terminal, and the management personnel manually determine whether the transmission status of the conveyor line to be detected at the target time is a congested state or an idle state.

[0044] Alternatively, based on the item transmission flow corresponding to the conveyor line to be detected, the method of determining the transmission status of the conveyor line to be detected at the target time may include: if the item transmission flow is less than a preset threshold, determining the congestion coefficient of the conveyor line to be detected at the target time; based on the congestion coefficient, determining the transmission status of the conveyor line to be detected at the target time.

[0045] The congestion coefficient is used to reflect the congestion degree of the items being transported on the conveyor line to be tested. The larger the congestion coefficient, the higher the congestion degree; the smaller the congestion coefficient, the lower the congestion degree.

[0046] In this embodiment, if the item transmission flow corresponding to the conveyor line to be detected is less than the preset threshold at the target time, there may be the following two reasons: 1. There are too many items on the conveyor line to be detected and congestion occurs, and the transmission state of the conveyor line to be detected is a congested state; 2. The number of items on the conveyor line to be detected is too small, and the number of items passing through the conveyor line to be detected in the unit time is small, that is, the conveyor line to be detected is in an idle state.

[0047] In order to more clearly and accurately determine the transmission status of the conveyor line to be detected at the target time, the congestion coefficient of the conveyor line to be detected at the target time can be determined, and the transmission status of the conveyor line to be detected at the target time can be determined based on the congestion coefficient. Specifically, the larger the congestion coefficient, the more congested the conveyor line to be detected is at the target time; the smaller the congestion coefficient, the more idle the conveyor line to be detected is at the target time.

[0048] In this embodiment, when the object transmission flow is less than a preset threshold, the congestion coefficient corresponding to the conveyor line to be detected is determined to quickly determine whether the conveyor line to be detected is in a congested state or an idle state, which is conducive to improving the accuracy of determining the transmission state.

[0049] In this embodiment, the method of determining the congestion coefficient of the conveyor line to be detected at the target time may include: determining the number of items transmitted by the conveyor line to be detected at the target time by visual counting. The more items there are, the higher the possibility of congestion in the conveyor line to be detected. The number of transmitted items can be determined as the congestion coefficient.

[0050] Alternatively, by pre-loading a pressure sensor on the conveying line to be detected, the pressure value that the conveying line to be detected is subjected to at the target time is determined, and the pressure value is determined as the congestion coefficient.

[0051] Alternatively, the congestion coefficient of the conveyor line to be detected at the target time is determined by: determining the total area of ​​the items transported in the video frame corresponding to the target video at the target time; determining the ratio of the total area to the conveyor line area of ​​the predetermined conveyor line to be detected, and determining the ratio as the congestion coefficient.

[0052] Specifically, the video frame corresponding to the target moment in the target video can be extracted, and the first contour information of each object in the video frame can be detected; based on the first contour information, the object area of ​​each object can be determined. Since the number, position, shape and other parameters of the conveyor lines to be detected are fixed parameters, the conveyor line area of ​​each conveyor line to be detected can be predetermined before extracting the video frame. For example, before extracting the video frame, the second contour information of each conveyor line to be detected in the target video is detected, and based on the second contour information, the conveyor line area of ​​each conveyor line to be detected can be determined.

[0053] For each conveyor line to be inspected, the sum of the areas of the items transported on the conveyor line to be inspected can be determined as the total area; and the ratio of the total area to the area of ​​the conveyor line to be inspected can be determined. A larger ratio means that the more items are transported on the conveyor line to be inspected, the more crowded the items are, and the higher the possibility of congestion; a smaller ratio means that the fewer items are transported on the conveyor line to be inspected, and the looser the conveyor line to be inspected is. Therefore, the ratio can be determined as the congestion coefficient.

[0054] This embodiment takes into account the actual situation of conveying items on the conveyor line. Since the sizes of items on the conveyor line to be detected are not the same, simply determining the congestion coefficient of the conveyor line to be detected by the number of items will lead to inaccurate congestion coefficients; and the larger the area of ​​the conveyor line to be detected occupied by the items, the more likely it is to be congested. Therefore, by using the ratio of the total area to the area of ​​the conveyor line as the congestion coefficient, the congestion situation of the conveyor line to be detected can be more accurately reflected.

[0055] In this embodiment, based on the congestion coefficient, determining the transmission state of the to-be-detected conveyor line at the target time includes: if the congestion coefficient is greater than a first preset coefficient, determining that the transmission state of the to-be-detected conveyor line at the target time is a congested state.

[0056] In this embodiment, by comparing the size of the congestion coefficient with the first preset coefficient, it can be determined whether the transmission state of the conveyor line to be detected is a congested state. Since the value range of the congestion coefficient is [0,1], the closer the congestion coefficient is to 1, the more congested the items on the conveyor line to be detected are. Therefore, the first preset coefficient can be set to a value close to 1; for example, the first preset coefficient can be set to 0.95. When the congestion coefficient is greater than 0.95, it is determined that the transmission state of the conveyor line to be detected at the target time is a congested state. By setting the first preset coefficient, this embodiment can quickly determine whether the transmission state of the conveyor line to be detected at the target time is a congested state.

[0057] In this embodiment, based on the congestion coefficient, determining the transmission state of the to-be-detected conveyor line at the target time includes: if the congestion coefficient is less than the second preset coefficient, determining that the transmission state of the to-be-detected conveyor line at the target time is an idle state.

[0058] Since the closer the congestion coefficient is to 0, the fewer items there are on the conveyor line to be detected, and the more idle the conveyor line to be detected is. Therefore, the second preset coefficient can be set to a value close to 0; for example, the second preset coefficient can be 0.5. If the congestion coefficient is less than 0.5, it can be determined that the transmission state of the conveyor line to be detected at the target time is idle. By setting the second preset coefficient, it can be quickly determined whether the conveyor line to be detected is in an idle state.

[0059] The technical solution of the embodiment of the present invention determines the item transmission flow rate corresponding to each conveyor line to be detected at the target time by acquiring a target video containing at least one conveyor line to be detected, wherein the item transmission flow rate is the number of items passing through the conveyor line to be detected per unit time; there is no need for detection personnel to manually determine the item transmission situation on the conveyor line to be detected; and, through the item transmission flow rate corresponding to each conveyor line to be detected, the transmission status of the conveyor line to be detected can be intuitively and effectively reflected, which is convenient for a comprehensive understanding of the item transportation situation in the logistics park, saves human resources, and provides convenience for resource scheduling in the logistics park.

[0060] Figure 2 It is a flowchart of another transmission status detection method provided according to an embodiment of the present invention. In this embodiment, based on the target video, the specific method of determining the object transmission flow corresponding to each conveyor line to be detected at the target time can be: based on the target video, detecting the target objects transmitted on each conveyor line to be detected at the target time; tracking the target objects on each conveyor line to be detected, and determining the transmission information of each target object within a preset time period; based on the transmission information, determining the object transmission flow corresponding to each conveyor line to be detected at the target time. Among them, the explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0061] like Figure 2 As shown, the method includes:

[0062] S210, acquiring a target video including at least one conveyor line to be detected.

[0063] S220: Based on the target video, detect the target object transmitted on each conveyor line to be detected at the target time.

[0064] The target object is an object transmitted on the conveyor line to be detected at the target time. Specifically, the video frame corresponding to the target time in the target video can be determined, and the target objects transmitted on each conveyor line to be detected in the video frame can be detected. The target detection model that has been trained in advance can be used to detect the target objects in the video frame.

[0065] In this embodiment, if there is a conveyor line to be detected in the target video, each target object in the video frame can be detected, and each target object can be determined as a target object transmitted on the unique conveyor line to be detected.

[0066] Specifically, when the number of conveyor lines to be detected contained in the target video cannot be determined, in order to distinguish the target items transmitted on each conveyor line to be detected so as to determine the transmission status of each conveyor line to be detected, based on the target video, a specific method for detecting the target items transmitted on each conveyor line to be detected at the target moment may include: determining the first contour information of at least one target item contained in the video frame corresponding to the target video at the target moment; for each target item, based on the first contour information of the target item and the predetermined second contour information of each conveyor line to be detected, determining the correspondence between the target item and each conveyor line to be detected; based on the correspondence, determining the target item transmitted on each conveyor line to be detected at the target moment.

[0067] In this embodiment, determining the first contour information of at least one target object contained in the video frame corresponding to the target video at the target moment includes: determining the first contour information of at least one target object contained in the video frame corresponding to the target video at the target moment based on a pre-trained target detection model.

[0068] The target detection model includes a YoLo model; for example, the S model in YoLoV8. The first contour information is used to reflect the edge contour of the detected target object. Exemplarily, if the shape of the target object is a cube, the contour reflected by the first contour information is a rectangle; if the shape of the target object is a cylinder, the contour reflected by the first contour information is a circle.

[0069] In a specific implementation, before determining the first contour information, the target detection model may be pre-trained. Specifically, sample data of the object may be obtained, and a deep learning model to be trained may be trained in a semi-supervised manner to obtain the target detection model.

[0070] This embodiment uses the YoLo model as the target detection model, which can ensure the accuracy of the detection results while minimizing the computational complexity.

[0071] In this embodiment, the second contour information of each conveyor line to be detected can be predetermined. Among them, the second contour information can reflect the edge contour of the conveyor line to be detected. In this embodiment, since the shape of the conveyor line to be detected is a fixed shape, the four vertices of the conveyor line to be detected can be pre-annotated in the form of manual annotation, and the convex polygon composed of the determined four vertices is used as the second contour information. Alternatively, a deep learning model can be used to obtain the edge contour of the conveyor line to be detected. Those skilled in the art can choose a way to determine the second contour information according to the specific circumstances, and the embodiment of the present invention is not limited to this.

[0072] In a specific implementation, for each target object, based on the first contour information and the second contour information, the specific implementation method of determining the correspondence between the target object and each conveyor line to be detected includes: for each target object corresponding to the first contour information, each conveyor line to be detected can be traversed to determine the second contour information containing the contour of the first contour information, and establish the correspondence between the conveyor line to be detected and the target object corresponding to the second contour information. Based on the correspondence between each target object and the conveyor line to be detected, the target object transmitted on each conveyor line to be detected at the target time is determined. Exemplarily, when determining the second contour information containing the contour of the first contour information, the method adopted may include: establishing a coordinate system based on a video frame, determining the vertex coordinates of each object of the target object in the coordinate system based on the first contour information, determining the conveyor line vertex coordinates of the conveyor line to be detected in the coordinate system based on the second contour information, and determining whether the conveyor line to be detected transmits the target object based on the object vertex coordinates and the conveyor line vertex coordinates.

[0073] This embodiment can quickly determine the target object transmitted on each conveyor line to be detected at the target time through the first contour information of the target object and the second contour information of the conveyor line to be detected, and accurately understand the corresponding relationship between the conveyor line to be detected and the target object, which is conducive to quickly determining the transmission status of each conveyor line to be detected.

[0074] In this embodiment, the second contour information corresponding to the conveyor line to be detected may be a convex polygon, and the correspondence between the target object and the conveyor line to be detected may be determined by determining whether the center point of the target object is within the convex polygon; if the center point of the target object is within the convex polygon corresponding to the conveyor line to be detected, then the target object corresponds to the conveyor line to be detected; if it is not within the convex polygon, then the target object does not correspond to the conveyor line to be detected.

[0075] In order to quickly determine the correspondence between the target object and the conveyor line to be detected, optionally, based on the first contour information of the target object and the predetermined second contour information of each conveyor line to be detected, the method of determining the correspondence between the target object and each conveyor line to be detected can be: determine the center point of the target object based on the first contour information; for each conveyor line to be detected, determine the correspondence between the target object and the conveyor line to be detected based on the point position information of the center point and the second contour information of the conveyor line to be detected.

[0076] Specifically, the geometric center of the cube corresponding to the first outline information can be determined as the center point of the target object. For example, if the outline reflected by the first outline information is a rectangle, the intersection of the diagonals of the rectangle is determined as the center point of the target object; if the outline reflected by the first outline information is a circle, the center of the circle is determined as the center point of the target object.

[0077] In a specific implementation, the point position information of the center point can be determined; wherein the point position information includes coordinates in a preset coordinate system; the preset coordinate system may include a rectangular coordinate system, a spherical coordinate system, and a polar coordinate system. The corresponding relationship between the target object and each conveyor line to be detected can be determined through the positional relationship between the point position information and the second contour information. The corresponding relationship includes that the target object is located on the conveyor line to be detected or the target object is located outside the conveyor line to be detected.

[0078] This embodiment determines the correspondence between the target object and each conveyor line to be inspected by determining the point position information of the center point and the second contour information, thereby simplifying the problem of determining the correspondence. The correspondence between the two can be determined without tedious calculations, thereby ensuring the accuracy of the results and improving the efficiency of determining the correspondence.

[0079] In this embodiment, based on the point position information of the center point and the second contour information, the specific implementation method of determining the correspondence between the target object and each conveyor line to be detected includes: determining the contour position information corresponding to the second contour information in the preset coordinate system. Based on the point position information and the contour position information, the positional relationship between the center point and the second contour information is determined, and based on the positional relationship, the correspondence between the target object and each conveyor line to be detected is determined.

[0080] The contour position information includes the coordinates corresponding to each contour point in the contour. The position relationship includes that the center point is located within the range corresponding to the second contour information, and that the center point is located outside the range corresponding to the second contour information.

[0081] Exemplarily, the preset coordinate system is a rectangular coordinate system, and the first point abscissa and the first point ordinate of the center point in the preset rectangular coordinate system can be determined; and the second point abscissa and the second point ordinate of each contour point corresponding to the second contour information in the rectangular coordinate system can be determined; because there are multiple contour points, based on the second point abscissa and the second point ordinate of each contour point, the minimum second point abscissa, the maximum second point abscissa, the minimum second point ordinate and the maximum second point ordinate can be determined. Determine whether the first point abscissa belongs to the first numerical range corresponding to the minimum second abscissa and the maximum second abscissa, and whether the first ordinate belongs to the second numerical range corresponding to the minimum second ordinate and the maximum second ordinate, and determine the positional relationship between the center point and the second contour information. Specifically, if the first point abscissa belongs to the first numerical range, and the first ordinate belongs to the second numerical range, then the positional relationship is determined to be that the center point is located within the range corresponding to the second contour information; if the first point abscissa does not belong to the first numerical range, or the first ordinate does not belong to the second numerical range, then the positional relationship is determined to be that the center point is located outside the range corresponding to the second contour information.

[0082] Furthermore, the correspondence between the target object and each conveyor line to be detected can be determined based on the positional relationship. Specifically, if the center point is within the range corresponding to the second contour information, it is determined that there is a correspondence between the target object and the conveyor line to be detected; if the center point is outside the range corresponding to the second contour information, it is determined that there is no correspondence between the target object and the conveyor line to be detected.

[0083] Alternatively, the area method can be used to determine whether the center point of the target object is within the convex polygon corresponding to the conveyor line to be detected, so as to determine the corresponding relationship between the target object and the conveyor line to be detected. Specifically, the specific implementation method of determining the corresponding relationship between the target object and the conveyor line to be detected includes: based on the second contour information of the conveyor line to be detected, determining the vertices of the conveyor line to be detected and the area of ​​the conveyor line; based on the point position information, determining at least one triangle formed by the center point and any two adjacent vertices of the conveyor line to be detected, determining the sum of the areas of each triangle, and if the sum of the areas is equal to the area of ​​the conveyor line, it is determined that there is a corresponding relationship between the target object and the conveyor line to be detected.

[0084] The convex polygon corresponding to the conveyor line to be detected may be a convex quadrilateral; and adjacent vertices represent two vertices connected by the sides of the convex quadrilateral.

[0085] In a specific implementation, the center point of the target object and two adjacent vertices can be used as the three vertices of a triangle, which are connected to form a triangle. The center point is connected to any two adjacent vertices of the convex quadrilateral of the conveyor line to be detected to determine the number of triangles formed; if the number of triangles is three, it means that the center point is on the edge of the convex quadrilateral of the conveyor line to be detected; if the number of triangles is four, the sum of the areas of the four triangles is determined. If the sum of the areas is equal to the area of ​​the conveyor line, it means that the center point is inside the convex quadrilateral of the conveyor line to be detected; if the sum of the areas is greater than the area of ​​the conveyor line, it means that the center point is outside the convex quadrilateral of the conveyor line to be detected. When the center point is inside the convex quadrilateral of the conveyor line to be detected, it means that there is a corresponding relationship between the target object and the conveyor line to be detected, that is, the conveyor line to be detected is used to transport the target object.

[0086] This embodiment determines the corresponding relationship between the target object and the conveyor line to be detected by the area method. The corresponding relationship can be accurately determined only by the center point of the target object and the vertex of the conveyor line to be detected. The determination process is convenient and efficient.

[0087] S230 , tracking the target objects on each conveyor line to be detected, and determining the transmission information of each target object within a preset time period.

[0088] In a specific implementation, after detecting the target object on the conveyor line to be detected, a unique identifier can be set for the target object, and based on the unique identifier and the target video, the transmission information of each target object within a preset time period can be determined. The transmission information can be the position information of the target object on the conveyor line to be detected.

[0089] In this embodiment, target objects on each conveyor line to be detected are tracked, and transmission information of each target object within a preset time period is determined, including: tracking target objects on each conveyor line to be detected based on a multi-target tracking algorithm, and determining transmission information of each target object within a preset time period.

[0090] The preset time period is a time period that takes the target time as the starting time and is separated from the starting time by a unit time.

[0091] Specifically, the target object can be tracked based on the target tracking algorithm. Exemplarily, the target tracking algorithm includes a SORT (Simple Online and Realtime Tracking) algorithm. The SORT algorithm is used to determine the target object to be tracked, and the operation speed is fast, and tracking can be achieved faster than real-time, which helps to quickly determine the transmission information, so as to timely determine the transmission status of the conveyor line to be detected based on the transmission information.

[0092] S240: Based on the transmission information, determine the object transmission flow rate corresponding to each conveyor line to be inspected at the target time.

[0093] In this embodiment, the movement of each target object within a preset time period can be determined by transmitting information to determine whether the target object passes through the conveyor line to be detected. The object transmission flow rate is determined based on the target number of target objects passing through the conveyor line to be detected within the preset time period.

[0094] Optionally, based on the transmission information, determine the item transmission flow rate corresponding to each conveyor line to be inspected at the target time, including: based on the transmission information, determine the target number of target items passing through each conveyor line to be inspected within a preset time period; for each conveyor line to be inspected, determine the target number corresponding to the conveyor line to be inspected as the item transmission flow rate corresponding to the target time.

[0095] It should be noted that the duration of the preset time period is a unit time, so the target number of target items passing through the preset time period can be directly determined as the item transmission flow. If the duration of the preset time period is not a unit time, the ratio of the duration of the preset time period to the unit time can be determined, and the determined target number can be divided by the ratio to determine the item transmission flow.

[0096] In this embodiment, the target quantity is determined as the item transmission flow rate, which simplifies the steps of determining the item transmission flow rate and helps to improve the efficiency of determining the item transmission flow rate.

[0097] S250. For each conveyor line to be detected, based on the article transmission flow corresponding to the conveyor line to be detected, determine the transmission state of the conveyor line to be detected at the target time.

[0098] In this embodiment, after determining the transmission state of the conveyor line to be detected at the target time, the loading rate of the conveyed items can be adjusted based on the transmission state of each conveyor line to be detected. Exemplarily, if the transmission state of the conveyor line to be detected at the target time is a congested state, the loading rate of the conveyor line to be detected is adjusted to the preset minimum rate until the transmission state of the conveyor line to be detected is a normal working state, and then the loading rate is restored to the default rate; if the transmission state of the conveyor line to be detected at the target time is an idle state, the loading rate of the conveyor line to be detected can be adjusted to the preset maximum rate until the transmission state of the conveyor line to be detected is a normal working state, and then the loading rate is restored to the default rate.

[0099] This embodiment detects the target items on the conveyor line to be detected at the target time and tracks the target items to determine the transmission information of the target items. The transmission information can be used to effectively count the target items passing through the conveyor line to be detected within a preset time period. This method is error-prone and highly efficient, and can quickly determine the item transmission flow rate.

[0100] The above describes in detail the embodiments corresponding to the transmission status detection method. In order to make the technical scheme of the present method further clear to those skilled in the art, a specific application scenario is given below by taking the conveyor line to be detected as a conveyor belt and the target item as a package as an example.

[0101] It should be noted that there are multiple conveyor belts to be detected in the logistics park, and the shape of the conveyor belts to be detected can be a convex polygon; illustratively, the shape of the conveyor belts to be detected is a convex quadrilateral, such as a rectangle.

[0102] Figure 3 is a schematic diagram of a transmission status detection process provided according to an embodiment of the present invention; Figure 3 As shown, in this embodiment, the detection of transmission status can be divided into the following steps: data labeling, hot zone calibration, detection model training, package counting, package flow determination, and congestion coefficient calculation. In this embodiment, the video captured by the camera in the logistics park is obtained as the target video, and the transmission status of the conveyor belt to be detected at the target time is determined based on the target video. The detailed description of each step is as follows:

[0103] Data labeling: The deep learning model to be trained can be trained in advance to obtain a target detection model for detecting and locating the package on the conveyor belt to be detected. Since the training model requires a large amount of labeling of the sample package data, a semi-supervised method can be used to label the data and train the deep learning model to be trained. Specifically, a labeling tool can be used to label a preset number of sample package data; for example, the preset number can be one-tenth of the data volume of the sample package data. Those skilled in the art can also determine the preset number according to the actual application situation, which is not limited in this embodiment. Further, when training the target detection model, the deep learning model to be trained can be based on the labeled data that has been labeled, and the deep learning model to be trained obtained by training the labeled data can be used to perform package detection on the unlabeled data, and the obtained detection results can be corrected and the deep learning model to be detected can be updated. The package detection of the unlabeled data is performed again based on the updated deep learning model to be detected, and this process is repeated until the accuracy of the detection effect of the deep learning model to be detected reaches a preset threshold, and then the target detection model can be obtained.

[0104] Hot zone calibration: The position of the conveyor belt to be detected can be determined based on the target video containing the conveyor belt to be detected captured by the camera in the logistics park. When there are multiple conveyor belts, it is necessary to calibrate the position of each conveyor belt, that is, the 4 points of the conveyor belt. In this embodiment, the 4 points can be determined by manual labeling, or by using a deep learning model for recognition, and the 4 points of the conveyor belt to be detected can be determined, which is not limited in this embodiment.

[0105] Detection model training: Considering the balance between detection accuracy and computational complexity, the target detection model can be determined as the YoLoV8 model. For example, the S model in YoLoV8 can be used as the target detection model. It should be noted that the difficulty of package detection in different scenarios varies, and different models can be selected as target detection models according to the complexity of the scenarios.

[0106] Package counting: When the conveyor belt to be detected is transmitting packages normally, the packages are constantly moving with the conveyor belt. When congestion occurs, the package transmission flow is approximately 0. The package transmission flow is the number of packages passing on the conveyor belt to be detected per unit time; for example, the unit time can be 5 seconds. To determine the package transmission flow, the SORT algorithm with low computational complexity can be used to track the packages on the conveyor belt to be detected. The SORT algorithm is based on Kalman filtering and Hungarian algorithm, and runs fast. It does not rely on GPU (graphics processing unit, microprocessor) devices and can achieve tracking speed faster than real time.

[0107] Since the counts of each conveyor belt are independent of each other, in order to determine the package transmission flow corresponding to each conveyor belt to be detected, the corresponding relationship between the package and the conveyor belt to be detected needs to be determined before tracking the package. Specifically, when determining the corresponding relationship, the center point corresponding to the package and the vertices of the quadrilateral corresponding to the conveyor belt to be detected can be determined. If the center point is within the quadrilateral corresponding to the conveyor belt to be detected, the center point belongs to the quadrilateral, and the package is transmitted on the conveyor belt to be detected corresponding to the quadrilateral; if the center point is not within the quadrilateral corresponding to the conveyor belt to be detected, the center point does not belong to the quadrilateral, and the package is not transmitted on the conveyor belt to be detected corresponding to the quadrilateral.

[0108] The area method can be used to determine whether the center point of the package is within the quadrilateral corresponding to the conveyor belt to be detected. Specifically, determine at least one triangle formed by the center point and any two adjacent vertices of the conveyor belt to be detected, and determine the sum of the areas of each triangle. If the sum of the areas is equal to the area of ​​the conveyor belt, it is determined that the package is transported on the conveyor belt to be detected corresponding to the quadrilateral.

[0109] Parcel flow determination: After tracking and counting the parcels, the number of parcels passing through the conveyor belt to be detected per unit time is determined as the parcel transmission flow.

[0110] Congestion coefficient calculation: A preset threshold value can be set in advance, for example, the preset threshold value is 1. If the package transmission flow is less than the preset threshold value, it can reflect that when the package on the conveyor belt to be detected is congested and squeezed, it is difficult for the package to pass through the conveyor belt to be detected, and the conveyor belt to be detected is in a congested state at the target time; or, it reflects that the number of packages on the conveyor belt to be detected is small, for example, there are no packages transported on the conveyor belt to be detected. In order to determine whether the conveyor belt to be detected is congested, it is necessary to continue to determine the congestion coefficient. Among them, the congestion coefficient is used to reflect the degree of congestion of the conveyor belt to be detected. In this embodiment, the total area of ​​each package on the conveyor belt to be detected can be determined as the proportion of the area of ​​the conveyor belt to be detected. If the congestion coefficient is greater than the first preset coefficient, it can be determined that the packages on the conveyor belt to be detected are congested; for example, the first preset coefficient can be set to 0.95.

[0111] Furthermore, a congestion warning can be performed based on the determined congestion coefficient. Exemplarily, when the congestion coefficient is greater than the first preset coefficient, an abnormal information can be generated and fed back to the management terminal to issue a congestion warning, and the loading rate of the conveyor belt to be detected is adjusted to the preset minimum rate. When the congestion coefficient is less than or equal to the first preset coefficient and is within the preset warning coefficient range, a first prompt information can be generated and fed back to the management terminal to indicate that there are a large number of packages on the conveyor belt to be detected and congestion will occur. When the congestion coefficient is less than the second preset coefficient, it means that there are a small number of packages on the conveyor belt to be detected, and a second prompt information can be generated and fed back to the management terminal, and the loading rate of the conveyor belt to be detected is adjusted to the preset maximum rate to avoid waste of resources.

[0112] This embodiment can output a quantified congestion coefficient, which can not only detect whether the conveyor belt to be detected is congested, but also issue an early warning based on the size of the congestion coefficient; and, by collecting target videos through the cameras in the logistics park itself, the hardware cost is reduced; this embodiment can simultaneously detect the transmission status of multiple conveyor belts to be detected, which provides convenience for resource allocation in the logistics park.

[0113] Figure 4 1 is a schematic diagram of the structure of a transmission status detection device provided according to an embodiment of the present invention, and the device is used to execute the transmission status detection method provided in any of the above embodiments. The device and the transmission status detection method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the transmission status detection device can refer to the embodiment of the above transmission status detection method. Figure 4 As shown, the device comprises:

[0114] A target video acquisition module 10 is used to acquire a target video containing at least one conveyor line to be detected;

[0115] The item transmission flow determination module 11 is used to determine the item transmission flow corresponding to each conveyor line to be detected at the target time based on the target video; wherein the item transmission flow is the number of items passing through the conveyor line to be detected per unit time;

[0116] The transmission status determination module 12 is used to determine the transmission status of each conveyor line to be detected at a target time based on the article transmission flow corresponding to the conveyor line to be detected.

[0117] Based on any optional technical solution in the embodiment of the present invention, optionally, the item transmission flow determination module 11 includes:

[0118] A target object detection submodule, for detecting the target object transmitted on each conveyor line to be detected at the target time based on the target video;

[0119] The transmission information determination submodule is used to track the target objects on each conveyor line to be detected and determine the transmission information of each target object within a preset time period;

[0120] The item transmission flow determination submodule is used to determine the item transmission flow corresponding to each conveyor line to be detected at a target time based on the transmission information.

[0121] Based on any optional technical solution in the embodiments of the present invention, optionally, the target object detection submodule includes:

[0122] A first contour information determining unit, configured to determine first contour information of at least one target object contained in a video frame corresponding to a target video at a target moment;

[0123] A corresponding relationship determining unit, for determining, for each target object, a corresponding relationship between the target object and each conveyor line to be detected based on the first contour information of the target object and the predetermined second contour information of each conveyor line to be detected;

[0124] The target object determination unit is used to determine the target object transmitted on each conveyor line to be detected at the target time based on the corresponding relationship.

[0125] Based on any optional technical solution in the embodiments of the present invention, optionally, the corresponding relationship determining unit includes:

[0126] A center point determination subunit, configured to determine a center point of a target object based on the first contour information;

[0127] The corresponding relationship determination subunit is used to determine the corresponding relationship between the target object and the conveyor line to be detected for each conveyor line to be detected based on the point position information of the center point and the second contour information of the conveyor line to be detected.

[0128] Based on any optional technical solution in the embodiments of the present invention, optionally, the corresponding relationship determination subunit includes:

[0129] A vertex determination subunit, used to determine the vertex and area of ​​the conveyor line to be detected based on the second contour information of the conveyor line to be detected;

[0130] The area sum determination subunit is used to determine at least one triangle formed by the center point and any two adjacent vertices of the conveyor line to be detected based on the point position information, and determine the area sum of each triangle. If the area sum is equal to the conveyor line area, it is determined that there is a corresponding relationship between the target object and the conveyor line to be detected.

[0131] Based on any optional technical solution in the embodiments of the present invention, optionally, the first profile information determining unit includes:

[0132] A first contour information determination subunit, configured to determine, based on a pre-trained target detection model, first contour information of at least one target object contained in a video frame corresponding to a target video at a target moment;

[0133] Among them, the target detection model includes the YoLo model.

[0134] Based on any optional technical solution in the embodiments of the present invention, optionally, the transmission information determination submodule includes:

[0135] The transmission information determination unit is used to track the target objects on each conveyor line to be detected based on a multi-target tracking algorithm, and determine the transmission information of each target object within a preset time period.

[0136] Based on any optional technical solution in the embodiments of the present invention, optionally, the item transmission flow determination submodule includes:

[0137] A target quantity determination unit, used to determine the target quantity of target objects passing through each conveyor line to be detected within a preset time period based on the transmission information;

[0138] The article transmission flow determination unit is used to determine, for each conveyor line to be detected, a target quantity corresponding to the conveyor line to be detected as the article transmission flow corresponding to the target time.

[0139] Based on any optional technical solution in the embodiment of the present invention, optionally, the transmission state determination module 12 includes:

[0140] A congestion coefficient determination submodule is used to determine the congestion coefficient of the conveyor line to be detected at the target time if the object transmission flow is less than a preset threshold; wherein the congestion coefficient is used to reflect the congestion degree of the objects transmitted on the conveyor line to be detected;

[0141] The transmission status determination submodule is used to determine the transmission status of the transmission line to be detected at a target time based on the congestion coefficient.

[0142] Based on any optional technical solution in the embodiments of the present invention, optionally, the congestion coefficient determination submodule includes:

[0143] a total area determination unit, used to determine the total area of ​​the objects transmitted in the video frame corresponding to the target video at the target time;

[0144] The ratio determination unit is used to determine the ratio of the total area to the predetermined conveying line area of ​​the conveying line to be detected, and determine the ratio as the congestion coefficient.

[0145] Based on any optional technical solution in the embodiments of the present invention, optionally, the transmission state determination submodule includes:

[0146] The congestion state determination unit is used to determine that the transmission state of the to-be-detected conveying line at the target time is a congestion state if the congestion coefficient is greater than a first preset coefficient.

[0147] Based on any optional technical solution in the embodiments of the present invention, optionally, the transmission state determination submodule includes:

[0148] The idle state determination unit is used to determine that the transmission state of the transmission line to be detected at the target time is an idle state if the congestion coefficient is less than the second preset coefficient.

[0149] The technical solution of the embodiment of the present invention determines the item transmission flow rate corresponding to each conveyor line to be detected at the target time by acquiring a target video containing at least one conveyor line to be detected, wherein the item transmission flow rate is the number of items passing through the conveyor line to be detected per unit time; there is no need for detection personnel to manually determine the item transmission situation on the conveyor line to be detected; and, through the item transmission flow rate corresponding to each conveyor line to be detected, the transmission status of the conveyor line to be detected can be intuitively and effectively reflected, which is convenient for a comprehensive understanding of the item transportation situation in the logistics park, saves human resources, and provides convenience for resource scheduling in the logistics park.

[0150] It is worth noting that in the embodiment of the above-mentioned transmission status detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0151] Figure 5 : is a schematic diagram of the structure of an electronic device that implements the transmission state detection method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0152] like Figure 5As shown, the electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 21 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 23. In RAM23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM22 and RAM23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0153] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0154] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs the various methods and processes described above, such as a transmission status detection method.

[0155] In some embodiments, the transmission status detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps of the transmission status detection method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the transmission status detection method in any other appropriate manner (e.g., by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0163] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A transmission status detection method, characterized in that: include: Acquire a target video containing at least one conveyor line to be detected; Based on the target video, determine the object transmission flow rate corresponding to each of the to-be-detected conveyor lines at the target time; wherein the object transmission flow rate is the number of objects passing through the to-be-detected conveyor line per unit time; For each of the conveyor lines to be detected, the transmission state of the conveyor line to be detected at the target time is determined based on the article transmission flow corresponding to the conveyor line to be detected.

2. The method according to claim 1, characterized in that The determining, based on the target video, the corresponding article transmission flow rate of each of the to-be-detected conveyor lines at the target time includes: Based on the target video, detecting the target object transmitted on each of the to-be-detected conveyor lines at the target time; Track the target objects on each conveyor line to be inspected, and determine the transmission information of each target object within a preset time period; Based on the transmission information, the object transmission flow rate corresponding to each of the conveyor lines to be detected at the target time is determined.

3. The method according to claim 2, characterized in that The detecting, based on the target video, the target object transmitted on each of the to-be-detected conveyor lines at the target time comprises: Determine first contour information of at least one of the target objects contained in a video frame corresponding to the target video at the target moment; For each of the target objects, based on the first contour information of the target object and the predetermined second contour information of each of the conveyor lines to be detected, determining the corresponding relationship between the target object and each of the conveyor lines to be detected; Based on the corresponding relationship, the target object transmitted on each of the conveyor lines to be detected at the target time is determined.

4. The method according to claim 3, characterized in that: The determining the correspondence between the target object and each of the conveyor lines to be detected based on the first contour information of the target object and the predetermined second contour information of each of the conveyor lines to be detected comprises: Determine the center point of the target object based on the first contour information; For each of the conveyor lines to be detected, the corresponding relationship between the target object and the conveyor line to be detected is determined based on the point position information of the center point and the second contour information of the conveyor line to be detected.

5. The method according to claim 4, characterized in that The determining the correspondence between the target object and the conveyor line to be detected based on the point position information of the center point and the second contour information of the conveyor line to be detected comprises: Based on the second contour information of the conveyor line to be detected, determining the vertices and the conveyor line area of ​​the conveyor line to be detected; Based on the point position information, at least one triangle formed by the center point and any two adjacent vertices of the conveyor line to be detected is determined, and the sum of the areas of each triangle is determined. If the sum of the areas is equal to the area of ​​the conveyor line, it is determined that there is the corresponding relationship between the target object and the conveyor line to be detected.

6. The method according to claim 3, characterized in that The determining of the first contour information of at least one target object contained in the video frame corresponding to the target video at the target moment includes: Determine, based on a pre-trained target detection model, first contour information of at least one target object contained in a video frame corresponding to the target video at the target moment; Wherein, the target detection model includes a YoLo model.

7. The method according to claim 2, characterized in that The tracking of target objects on each conveyor line to be detected and determining the transmission information of each target object within a preset time period include: Based on a multi-target tracking algorithm, the target objects on each of the conveyor lines to be detected are tracked to determine the transmission information of each of the target objects within a preset time period.

8. The method according to claim 2, characterized in that: The determining, based on the transmission information, the corresponding article transmission flow rate of each of the to-be-detected conveyor lines at the target time includes: Based on the transmission information, determining a target number of target objects passing through each of the to-be-detected conveyor lines within the preset time period; For each of the conveyor lines to be detected, the target quantity corresponding to the conveyor line to be detected is determined as the article transfer flow corresponding to the target time.

9. The method according to claim 1, characterized in that: The determining the transmission state of the conveyor line to be detected at the target time based on the article transmission flow corresponding to the conveyor line to be detected includes: If the object transmission flow is less than a preset threshold, the congestion coefficient of the to-be-detected conveyor line at the target time is determined; wherein the congestion coefficient is used to reflect the congestion degree of the objects transmitted on the to-be-detected conveyor line; Based on the congestion coefficient, the transmission state of the to-be-detected conveying line at the target time is determined.

10. The method according to claim 9, characterized in that The determining of the congestion coefficient of the to-be-detected conveying line at the target time comprises: Determine the total area of ​​the object transmitted in the video frame corresponding to the target video at the target time; The ratio of the total area to a predetermined conveying line area of ​​the conveying line to be detected is determined, and the ratio is determined as the congestion coefficient.

11. The method according to claim 9, characterized in that The determining, based on the congestion coefficient, the transmission state of the to-be-detected transmission line at the target time comprises: If the congestion coefficient is greater than the first preset coefficient, it is determined that the transmission state of the to-be-detected conveying line at the target time is a congestion state.

12. The method according to claim 9, characterized in that The determining, based on the congestion coefficient, the transmission state of the to-be-detected transmission line at the target time comprises: If the congestion coefficient is less than the second preset coefficient, it is determined that the transmission state of the to-be-detected conveying line at the target time is an idle state.

13. A transmission status detection device, characterized in that: include: A target video acquisition module, used to acquire a target video containing at least one conveyor line to be detected; An item transmission flow determination module is used to determine the item transmission flow corresponding to each of the to-be-detected conveyor lines at a target time based on the target video; wherein the item transmission flow is the number of items passing through the to-be-detected conveyor line per unit time; The transmission status determination module is used to determine the transmission status of each of the conveyor lines to be detected at the target time based on the article transmission flow corresponding to the conveyor line to be detected.

14. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the transmission status detection method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transmission status detection method according to any one of claims 1 to 12 when executed.

Images