Storage frame state detection method, device, control equipment, system and storage medium

Abstract

The present application provides a storage frame state detection method, device, control equipment, system and storage medium, the method comprising: obtaining a first state image of the storage frame opening captured by a camera device; detecting whether the first state image includes a full frame detection mark; if the first state image does not include a full frame detection mark, measuring the distance of the package in the storage frame, and determining the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than the reference vertical distance from the camera device to the horizontal plane where the storage frame opening is located, determining that the storage frame is in a full frame state. The implementation scheme of the present application can effectively solve the problem of inaccurate photoelectric detection of the storage frame state, improve the effective utilization rate of the storage frame, reasonably plan the sorting plan, and reduce the cost of manpower maintenance.

Description

Technical Field

[0001] The present application relates to the field of logistics technology, and in particular to a material storage frame status detection method, device, control equipment, system and storage medium. Background Art

[0002] In recent years, machine vision algorithms have been increasingly used in the field of logistics sorting. The most basic feature of machine vision systems is to improve production flexibility and automation. In some dangerous working environments that are not suitable for manual operations or where manual vision is difficult to meet the requirements, machine vision is often used to replace manual vision. At the same time, in large-scale repetitive industrial production processes, the use of machine vision detection methods can greatly improve production efficiency and automation.

[0003] In the field of logistics sorting, parcels are dropped into the storage bins at designated slots through sorting control. Currently, most existing technologies rely on photoelectric sensors to sense whether the storage bins are full. When the storage bin is detected to be full, the main control system will issue a command to replace the empty storage bins, and the sorting task can be continued. However, this method that relies on photoelectric sensors cannot sense the status information of the object during transmission in real time, and the detection of photoelectric sensors is not accurate enough. Summary of the invention

[0004] The present application provides a storage frame status detection method, device, control equipment, system and storage medium, which can effectively solve the problem of inaccurate photoelectric detection of storage frame status, improve the effective utilization rate of the storage frame, reasonably plan the sorting plan, and reduce manpower maintenance costs.

[0005] On the one hand, the present application provides a material storage frame state detection method, which is applied to a sorting system, wherein the sorting system includes a material storage frame and a visual detection device, wherein the visual detection device includes a camera device and a control device, wherein the camera device is arranged at a preset height above an opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined surface, wherein the angle between the inclined surface and the horizontal line at the bottom of the camera device is an acute angle, and a full frame detection mark is arranged on the inner wall of the inclined surface, wherein the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located, and the method execution body is the control device, and the method includes:

[0006] Acquire a first state image of the opening of the material storage frame captured by a camera device;

[0007] Detecting whether the first state image includes a full frame detection mark;

[0008] If the first state image does not include a full frame detection mark, measuring the distance of the package in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the material storage frame is located;

[0009] If the first vertical distance is smaller than a reference vertical distance from the camera device to the horizontal plane where the material storage frame opening is located, it is determined that the material storage frame is in a full frame state.

[0010] In some embodiments of the present application, the detecting whether the first state image includes a full frame detection mark includes:

[0011] Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance;

[0012] According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images;

[0013] It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

[0014] In some embodiments of the present application, the step of measuring the distance between the packages in the storage frame and determining a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the storage frame is located includes:

[0015] Get the preset package width range information;

[0016] Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information;

[0017] Obtaining predetermined focal length information of a camera device;

[0018] A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

[0019] In some embodiments of the present application, calculating the basic actual width information of the package in the field of view of the camera device according to the package width range information includes:

[0020] Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range;

[0021] The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

[0022] In some embodiments of the present application, the method further comprises:

[0023] If the first state image includes a full frame detection mark, measuring the distance of the package in the material storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located;

[0024] If the second vertical distance is smaller than the reference vertical distance, acquiring a second state image of the material storage frame opening captured by a camera device;

[0025] Detecting whether the second state image includes a full frame detection mark;

[0026] If the second state image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, an abnormal prompt message is sent to the preset user terminal.

[0027] On the other hand, the present application provides a sorting system, characterized in that the sorting system includes a material storage frame and a visual detection device, the visual detection device includes a camera device and a control device, the camera device is arranged at a preset height above the opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined surface, the angle between the inclined surface and the horizontal line of the bottom of the camera device is an acute angle, the inner wall of the inclined surface is provided with a full frame detection mark, and the full frame detection mark is a preset distance from the horizontal plane where the opening of the material storage frame is located;

[0028] The camera device is used to collect a first state image of the opening of the material storage frame and transmit the first state image to the control device;

[0029] The control device is used to receive the first state image captured and transmitted by the camera device; detect whether the first state image includes a full frame detection mark; if the first state image does not include the full frame detection mark, measure the distance of the package in the storage frame to determine the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than the reference vertical distance from the camera device to the horizontal plane where the opening of the storage frame is located, determine that the storage frame is in a full frame state.

[0030] In some embodiments of the present application, the control device is specifically used for:

[0031] Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance;

[0032] According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images;

[0033] It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

[0034] In some embodiments of the present application, the control device is specifically used for:

[0035] Get the preset package width range information;

[0036] Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information;

[0037] Obtaining predetermined focal length information of a camera device;

[0038] A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

[0039] In some embodiments of the present application, the control device is specifically used for:

[0040] Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range;

[0041] The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

[0042] In some embodiments of the present application, the control device is also used for:

[0043] If the first state image includes a full frame detection mark, measuring the distance of the package in the material storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located;

[0044] If the second vertical distance is smaller than the reference vertical distance, acquiring a second state image of the material storage frame opening captured by the camera device;

[0045] Detecting whether the second state image includes a full frame detection mark;

[0046] If the second state image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, an abnormal prompt message is sent to the preset user terminal.

[0047] On the other hand, the present application further provides a material storage frame state detection device, which is applied to a sorting system, wherein the sorting system comprises a material storage frame and a visual detection device, wherein the visual detection device comprises a camera device and a control device, wherein the camera device is arranged at a preset height above an opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined surface, wherein the angle between the inclined surface and the horizontal line at the bottom of the camera device is an acute angle, and a full frame detection mark is arranged on the inner wall of the inclined surface, wherein the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located, and the material storage frame state detection device is located in the control device, and the device comprises:

[0048] An image acquisition module, used to acquire a first state image of the material storage frame opening captured by a camera device;

[0049] A mark detection module, used to detect whether the first state image includes a full frame detection mark;

[0050] A state detection module is used to measure the distance of the package in the storage frame if the first state image does not include a full frame detection mark, and determine a first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than a reference vertical distance from the camera device to the horizontal plane where the opening of the storage frame is located, it is determined that the storage frame is in a full frame state.

[0051] In some embodiments of the present application, the marker detection module is specifically used for:

[0052] Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance;

[0053] According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images;

[0054] It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

[0055] In some embodiments of the present application, the state detection module is specifically used to:

[0056] Get the preset package width range information;

[0057] Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information;

[0058] Obtaining predetermined focal length information of a camera device;

[0059] A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

[0060] In some embodiments of the present application, the state detection module is specifically used to:

[0061] Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range;

[0062] The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

[0063] In some embodiments of the present application, the state detection module is further used to: if the first state image includes a full frame detection mark, measure the distance of the package in the storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located;

[0064] The image acquisition module is also used to acquire a second state image of the material storage frame opening captured by the camera device if the second vertical distance is less than the reference vertical distance;

[0065] The mark detection module is further used to detect whether the second state image includes a full frame detection mark;

[0066] The status detection module is also used to: if the second status image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, send an abnormal prompt message to a preset user terminal.

[0067] On the other hand, the present application also provides a control device, the control device comprising:

[0068] one or more processors;

[0069] Memory; and

[0070] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the material storage frame status detection method.

[0071] On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in the material storage frame status detection method.

[0072] In the case where the prior art photoelectric detection of the state of the material storage frame is inaccurate, the present application uses a visual detection device in combination with the design of the inclined surface shape on one side of the material storage frame, and a full frame detection mark is set on the inner wall of the inclined surface of the material storage frame. When the visual detection device cannot detect the full frame detection mark, the vertical distance between the camera device in the material storage frame and the horizontal plane where the vertex of the package area in the material storage frame is located determines whether the material storage frame is in a full frame state; the combination of the two aspects can effectively solve the problem of inaccurate photoelectric detection of the state of the material storage frame, improve the effective utilization rate of the material storage frame, reasonably plan the sorting plan, and reduce the manpower maintenance cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 is a schematic diagram of a scene of a sorting system provided in an embodiment of the present application;

[0075] Figure 2 It is a schematic diagram of an embodiment of a method for detecting the state of a material storage frame provided in an embodiment of the present application;

[0076] Figure 3 is a flowchart of an embodiment of step 202 provided in an embodiment of the present application;

[0077] Figure 4 It is a schematic structural diagram of an embodiment of a material storage frame state detection device provided in an embodiment of the present application;

[0078] Figure 5 It is a schematic diagram of the structure of an embodiment of the control device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0080] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0081] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0082] The embodiments of the present application provide a material storage frame status detection method, device, control equipment, system and storage medium, which are described in detail below.

[0083] See also Figure 1 , Figure 1 Schematic diagram of a scene of a sorting system provided in an embodiment of the present application. The sorting system may include a control device 100, a camera device 200 connected to the control device network, and a storage frame 101 below the camera device 200. The storage frame 101 may be used to store packages 102. The control device 100 is integrated with a storage frame state detection device, such as Figure 1 Control devices in.

[0084] The camera device may be an industrial camera, such as a standard industrial camera. Specifically, the industrial camera may be a short-focus USB camera with 480p, a frame rate of 30fps, a lens focal length of 3mm, and a camera mounted at a height of about 550mm to cover an area of ​​650mmx650mm.

[0085] In order to better assist visual image algorithms, such as Figure 1 As shown, in the structural design of the material storage frame, one side of the material storage frame is a slope, and the angle between the slope and the horizontal line of the bottom of the camera device is an acute angle (the size of the acute angle can be flexibly set depending on the actual usage scenario). For example, the opening of the material storage frame is designed to be a semi-trumpet mouth (the cross section is semi-trumpet-shaped), or an inverted trapezoidal mouth (the cross section is an inverted trapezoid), etc., so that the camera of the camera device can capture the image information of the inner wall of the material storage frame, so that a unique full frame mark can be designed on the inner wall of the material storage frame. Specifically, in the embodiment of the present application, a full frame detection mark 103 is provided on the inner wall of the slope of the material storage frame, and the full frame detection mark is a preset distance from the horizontal plane where the material storage frame opening 104 is located (which can be adjusted according to the field of view coverage of the camera device).

[0086] Among them, the full frame detection mark 103 can be a paint layer that is different from the color of the inner wall of the storage frame 101, or a sticker or fixed marking accessory (such as a welded metal plate) that is different from the color of the inner wall of the storage frame 101, etc., as long as it can serve as a marking, and is not limited here.

[0087] It should be noted that in order to effectively detect whether the material storage frame 101 is in a full frame state, the full frame detection mark 103 is at a preset distance from the horizontal plane where the material storage frame opening 104 is located, which is preferably less than 1 / 5 of the depth of the material storage frame. At this time, the full frame detection mark 103 is on the inclined surface of the material storage frame, closer to the horizontal plane of the material storage frame opening, to avoid being blocked by wrapping.

[0088] The camera device 200 in the embodiment of the present application is used to capture the status image of the opening of the material storage frame and transmit the status image to the control device 100.

[0089] In the embodiment of the present application, the control device 100 is mainly used to receive a status image captured and transmitted by the camera device 200; detect whether the status image includes a full frame detection mark; if the status image does not include a full frame detection mark, measure the distance of the package in the storage frame 101 to determine a first vertical distance between the camera device 200 and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than a reference vertical distance from the camera device 200 to the horizontal plane where the opening of the storage frame 101 is located, it is determined that the storage frame 101 is in a full frame state. Since the vertical distance from the camera device 200 to the horizontal plane where the opening of the storage frame 101 is located is unchanged, the distance can be obtained in advance and defined as the reference vertical distance for comparison.

[0090] It should be noted that, in the embodiment of the present application, the camera device needs to be set facing the opening of the material storage frame and at a preset height above the material storage frame. The preset height can be determined in combination with the field of view coverage of the camera device to ensure that the field of view coverage of the camera device covers the opening of the material storage frame. Only in this way can the camera device play an actual visual detection role.

[0091] Specifically, the control device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the control device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0092] It is understandable that the control device 100 used in the embodiment of the present application may also be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific control device 100 may be a desktop terminal or a mobile terminal, for example, it may be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0093] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer control devices as shown in, e.g. Figure 1 Only one visual inspection device (a camera device and a control device) and one storage frame are shown. It can be understood that the sorting system can also include one or more other visual inspection devices (a camera device and a control device) and storage frames. The visual inspection devices and storage frames can be arranged in a one-to-one correspondence. Of course, it can also include a control device and multiple camera devices, and multiple camera devices and multiple storage frames can be arranged in a one-to-one correspondence. The specific details are not limited here.

[0094] In some embodiments of the present application, the camera device is used to capture a first state image at the opening of the material storage frame and transmit the first state image to the control device; the control device is used to receive the first state image captured and transmitted by the camera device; detect whether the first state image includes a full frame detection mark; if the first state image does not include a full frame detection mark, measure the distance to the package in the material storage frame to determine a first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located; if the first vertical distance is less than the reference vertical distance, determine that the material storage frame is in a full frame state.

[0095] Among them, the vertex of the package area can be the vertex of the current package in the material storage frame. In other embodiments of the present application, when detecting the full frame detection mark, the first state image of the material storage frame opening can also be segmented into multiple regions (see description in subsequent embodiments). Therefore, the first state image can be divided into multiple package areas, and the vertex of the package area can be the vertex of the package in one of the package areas.

[0096] For example, the multiple package areas include package area 1, package area 2, package area 3 and package area 4, then the package area vertex can be the vertex packaged in one of the package areas, for example, the vertex packaged in package area 1, that is, the highest point of the package in package area 1. If the first vertical distance between the camera device and the horizontal plane where the vertex packaged in package area 1 is located is less than the reference vertical distance, it is determined that the material storage frame is in a full frame state. If the first vertical distance between the camera device and the horizontal plane where the vertex packaged in package area 1 is located is greater than the reference vertical distance, the vertical distance between the vertex packaged in other package areas (such as package area 2, package area 3, package area 4, etc.) and the horizontal plane where the material storage frame opening is located can be further determined, and then compared with the reference vertical distance. As long as the distance between the camera device and the horizontal plane where the vertex packaged in one of the package areas is located is less than the reference vertical distance, it is determined that the material storage frame is in a full frame state.

[0097] In some embodiments of the present application, the control device is specifically used to: determine the image segmentation size information based on the pre-acquired storage frame size or minimum package size information; perform multi-region segmentation on the first state image according to the image segmentation size information to obtain multiple segmented images; respectively detect whether the multiple segmented images include a full frame detection mark, and if the multiple segmented images include a full frame detection mark, determine that the first state image includes a full frame detection mark.

[0098] In some embodiments of the present application, the control device is specifically used to: obtain preset package width range information;

[0099] According to the package width range information, calculate the basic actual width information of the package in the field of view of the camera device; obtain the predetermined focal length information of the camera device; according to the basic actual width information and the focal length information of the camera device, calculate the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located.

[0100] In some embodiments of the present application, the control device is specifically used to: calculate the average value of the maximum value of the package width range and the minimum value of the package width range in the package width range; use the average value as the basic actual width of the package in the field of view of the camera device, and obtain the basic actual width information of the package in the field of view of the camera device.

[0101] In some embodiments of the present application, the control device is also used to: if the first state image includes a full frame detection mark, measure the distance of the package in the storage frame, and determine the second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the second vertical distance is less than the reference vertical distance, obtain a second state image of the storage frame opening captured by the camera device, and the acquisition time of the second state image is after the acquisition time of the first state image; detect whether the second state image includes a full frame detection mark; if the second state image includes a full frame detection mark, and the current third vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located is still less than the reference vertical distance, send an abnormal prompt information to a preset user terminal.

[0102] In addition, if Figure 1 As shown, the sorting system may further include a memory 300 for storing material storage frame data, such as image data of the material storage frame opening state and full frame state data of the material storage frame collected by a camera device.

[0103] It should be noted that Figure 1 The scenario diagram of the sorting system shown is only an example. The sorting system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the sorting system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0104] First, an embodiment of the present application provides a material storage frame state detection method, which is applied to a sorting system, wherein the sorting system includes a material storage frame and a visual detection device, wherein the visual detection device includes a camera device and a control device, wherein the camera device is arranged at a preset height above an opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined surface, and the angle between the inclined surface and the horizontal line of the bottom of the camera device is an acute angle, and a full frame detection mark is provided on the inner wall of the inclined surface, and the full frame detection mark is a preset distance from the horizontal plane where the opening of the material storage frame is located, and the method execution subject is the control device.

[0105] like Figure 2 As shown, it is a schematic flow chart of an embodiment of a method for detecting the state of a material storage frame in an embodiment of the present application. The method for detecting the state of a material storage frame includes the following steps 201 to 204:

[0106] 201. Acquire a first state image of an opening of a material storage frame captured by a camera device.

[0107] 202. Detect whether the first state image includes a full frame detection mark.

[0108] 203. If the first state image does not include a full frame detection mark, measure the distance of the package in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the material storage frame is located.

[0109] 204. If the first vertical distance is less than a reference vertical distance from the camera device to the horizontal plane where the material storage frame opening is located, determine that the material storage frame is in a full frame state.

[0110] In the case where the prior art photoelectric detection of the state of the material storage frame is inaccurate, the present application uses a visual detection device in combination with the design of the inclined surface shape on one side of the material storage frame, and a full frame detection mark is set on the inner wall of the inclined surface of the material storage frame. When the visual detection device cannot detect the full frame detection mark, the vertical distance between the camera device in the material storage frame and the horizontal plane where the vertex of the package area in the material storage frame is located determines whether the material storage frame is in a full frame state; the combination of the two aspects can effectively solve the problem of inaccurate photoelectric detection of the state of the material storage frame, improve the effective utilization rate of the material storage frame, reasonably plan the sorting plan, and reduce the manpower maintenance cost.

[0111] Since the full-frame detection mark is relatively small relative to the entire storage frame, the applicant has determined, based on actual usage, that there are certain detection errors when detecting the full-frame detection mark based on the entire image, resulting in inaccurate results and a slow detection rate. Therefore, in some implementation schemes of the present application, the image can be segmented and then the full-frame detection mark detection can be performed separately to improve the detection accuracy and detection rate. Specifically, Figure 3As shown, the step 202 of detecting whether the first state image includes a full frame detection mark may further include the following steps 301 to 303:

[0112] 301. Determine image segmentation size information according to the pre-acquired material storage frame size or minimum package size information.

[0113] Among them, the material storage frame size or the minimum package size information can be obtained in advance. Since the image segmentation size is too large, it has no practical effect. Since ranging is required later based on the segmented image, if the segmentation size is smaller than the minimum package size, subsequent ranging cannot be achieved. Therefore, here, the image segmentation size information is determined based on the material storage frame size or the minimum package size information obtained in advance. At least it is necessary to ensure that the image segmentation size is smaller than the material storage frame size and larger than the minimum package size. It is also possible to detect the position of the package in the first state image based on the target detection algorithm, determine the image segmentation size information based on the position of the package, and cut based on the position of the package during subsequent cutting to ensure that the package in each segmented image is not cut open. Each segmented image includes a package image to facilitate subsequent ranging.

[0114] 302. Perform multi-region segmentation on the first state image according to the image segmentation size information to obtain a plurality of segmented images.

[0115] 303 . Detect respectively whether the multiple segmented images include a full-frame detection mark. If the multiple segmented images include a full-frame detection mark, determine that the first state image includes the full-frame detection mark.

[0116] In some embodiments of the present application, the step 203 of measuring the distance of the packages in the storage frame to determine the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located includes: obtaining preset package width range information; calculating the basic actual width information of the package in the field of view of the camera device based on the package width range information; obtaining the predetermined focal length information of the camera device; and calculating the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located based on the basic actual width information and the focal length information of the camera device.

[0117] Among them, the package width range information is the information of the package width range, for example, the package width range is Wmin~Wmax, wherein Wmin is the minimum width of the package, and Wmax is the maximum width of the package. The package width range information can be pre-determined according to the size of the package to be sorted, and the package width range is directly used later.

[0118] Specifically, calculating the basic actual width information of the package in the field of view of the camera device based on the package width range information includes: calculating the average value of the maximum value of the package width range and the minimum value of the package width range in the package width range; using the average value as the basic actual width of the package in the field of view of the camera device, to obtain the basic actual width information of the package in the field of view of the camera device.

[0119] This application mainly uses similar triangles to calculate the first vertical distance between the camera device and the horizontal plane where the vertices of the package area in the storage frame are located. In a specific embodiment, the control device can obtain the package size range {(Wmin, Hmin, Zmin) ~ (Wmax, Hmax, Zmax)}, and the basic actual width of the package in the camera field of view can be defined as W = (Wmin + Wmax) / 2 according to the package size range. In this application, a target object can be manually placed in advance at a storage frame position D above the camera device, and the target object can be photographed with a camera device and the pixel width P of the object can be measured. The formula for the focal length of the camera device can be obtained: F = (P*D) / W, which is entered into the control device. The focal length information of the camera device can be directly obtained later. When the package is close to the camera device, similar triangles are used to calculate the first vertical distance between the camera device and the horizontal plane where the vertices of the package area in the storage frame are located: D' = (W*F) / P. Finally, the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located can be approximately estimated, where P refers to the pixel distance, W is the basic width of the package, and F is the focal length of the camera.

[0120] It should be noted that if the first state image is segmented into multiple regions in the above step 302, the multiple segmented images obtained include a package image, and the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located is calculated based on the basic actual width information and the focal length information of the camera device. It can be that the vertical distances between the camera device and the vertex of the package area in multiple separated images are calculated separately to obtain multiple vertical distances, and the minimum vertical distance among the multiple vertical distances is determined as the first vertical distance between the package in the storage frame and the camera device.

[0121] In some special scenarios, for example, when packages are piled up on one side of the storage frame, a full frame detection mark may be detected, but the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located is less than the reference vertical distance. At this time, it can be further determined by detection whether there is such an abnormal situation. If so, the management personnel are reminded to handle it. Therefore, the method in the embodiment of the present application may also include:

[0122] If the first state image includes a full frame detection mark, measuring the distance of the package in the material storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located;

[0123] If the second vertical distance is smaller than the reference vertical distance, acquiring a second state image of the material storage frame opening captured by the camera device;

[0124] Detect whether the second-state image includes a full-frame detection mark; if the second-state image includes a full-frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, send an abnormal prompt message to the preset user terminal.

[0125] In order to better implement the material storage frame state detection method in the embodiment of the present application, on the basis of the material storage frame state detection method, a material storage frame state detection device is also provided in the embodiment of the present application, which is applied to a sorting system, wherein the sorting system comprises a material storage frame and a visual detection device, wherein the visual detection device comprises a camera device and a control device, wherein the camera device is arranged at a preset height directly above the opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined surface, and the angle between the inclined surface and the horizontal line of the bottom of the camera device is an acute angle, and a full frame detection mark is arranged on the inner wall of the inclined surface, and the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located, and the material storage frame state detection device is located in the control device.

[0126] like Figure 4 As shown, the material storage frame state detection device 400 includes:

[0127] An image acquisition module 401 is used to acquire a first state image of the material storage frame opening captured by a camera device;

[0128] A mark detection module 402, used to detect whether the first state image includes a full frame detection mark;

[0129] A state detection module is used to measure the distance of the package in the storage frame if the first state image does not include a full frame detection mark, and determine a first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than a reference vertical distance, determine that the storage frame is in a full frame state.

[0130] In the case where the prior art photoelectric detection of the state of the material storage frame is inaccurate, the present application uses a visual detection device in combination with the design of the inclined surface shape on one side of the material storage frame, and a full frame detection mark is set on the inner wall of the inclined surface of the material storage frame. When the visual detection device cannot detect the full frame detection mark, the vertical distance between the camera device in the material storage frame and the horizontal plane where the vertex of the package area in the material storage frame is located determines whether the material storage frame is in a full frame state; the combination of the two aspects can effectively solve the problem of inaccurate photoelectric detection of the state of the material storage frame, improve the effective utilization rate of the material storage frame, reasonably plan the sorting plan, and reduce the manpower maintenance cost.

[0131] In some embodiments of the present application, the marker detection module 402 is specifically used to:

[0132] Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance;

[0133] According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images;

[0134] It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

[0135] In some embodiments of the present application, the state detection module 403 is specifically used to:

[0136] Get the preset package width range information;

[0137] Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information;

[0138] Obtaining predetermined focal length information of a camera device;

[0139] A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

[0140] In some embodiments of the present application, the state detection module 403 is specifically used to:

[0141] Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range;

[0142] The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

[0143] In some embodiments of the present application, the state detection module 403 is further used to: if the first state image includes a full frame detection mark, measure the distance of the package in the storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located;

[0144] The image acquisition module 401 is further configured to acquire a second state image of the opening of the material storage frame captured by the camera device if the second vertical distance is less than a preset reference vertical distance from the camera device to the upper opening of the material storage frame;

[0145] The mark detection module 402 is further used to detect whether the second state image includes a full frame detection mark;

[0146] The state detection module 403 is also used to: if the second state image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, send an abnormal prompt message to a preset user terminal.

[0147] The embodiment of the present invention further provides a control device, which integrates any one of the material storage frame state detection devices provided in the embodiment of the present invention, and the control device includes:

[0148] one or more processors;

[0149] Memory; and

[0150] One or more applications, wherein the one or more applications are stored in the memory and are configured to execute, by the processor, the steps of the material storage frame status detection method described in any of the above material storage frame status detection method embodiments.

[0151] The embodiment of the present invention further provides a control device, which integrates any one of the material storage frame state detection devices provided in the embodiment of the present invention. Figure 5 As shown, it shows a schematic diagram of the structure of the control device involved in the embodiment of the present invention, specifically:

[0152] The control device may include one or more processing core processors 501, one or more computer-readable storage media memories 502, a power supply 503, an input unit 504 and other components. Those skilled in the art will appreciate that Figure 5 The control device structure shown in the figure does not constitute a limitation on the control device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] The processor 501 is the control center of the control device, and uses various interfaces and lines to connect various parts of the entire control device. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, the processor 501 performs various functions of the control device and processes data, thereby monitoring the control device as a whole. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0154] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the control device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0155] The control device also includes a power supply 503 for supplying power to each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0156] The control device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0157] Although not shown, the control device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the control device will load the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application programs stored in the memory 502, thereby realizing various functions, as follows:

[0158] Acquire a first state image of the opening of the material storage frame captured by a camera device;

[0159] Detecting whether the first state image includes a full frame detection mark;

[0160] If the first state image does not include a full frame detection mark, measuring the distance of the package in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the material storage frame is located;

[0161] If the first vertical distance is smaller than the reference vertical distance, it is determined that the material storage frame is in a full frame state.

[0162] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0163] To this end, an embodiment of the present invention provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the material storage frame status detection methods provided in the embodiments of the present invention. For example, the computer program may be loaded by a processor to execute the following steps:

[0164] Acquire a first state image of the opening of the material storage frame captured by a camera device;

[0165] Detecting whether the first state image includes a full frame detection mark;

[0166] If the first state image does not include a full frame detection mark, measuring the distance of the package in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the material storage frame is located;

[0167] If the first vertical distance is smaller than the reference vertical distance, it is determined that the material storage frame is in a full frame state.

[0168] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, which will not be repeated here. In specific implementation, the above units or structures can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0169] The above is a detailed introduction to a material storage frame status detection method, device, control equipment, system and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting the state of a material storage frame, characterized in that: The invention is applied to a sorting system, wherein the sorting system comprises a material storage frame and a visual detection device, wherein the visual detection device comprises a camera device and a control device, wherein the camera device is arranged at a preset height above the opening of the material storage frame, and the control device is connected to the camera device through a network; one side of the material storage frame is an inclined plane, wherein the angle between the inclined plane and the horizontal line at the bottom of the camera device is an acute angle, and a full frame detection mark is arranged on the inner wall of the inclined plane, wherein the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located, and the execution subject of the method is the control device, and the method comprises: Acquire a first state image of the opening of the material storage frame captured by a camera device; Detecting whether the first state image includes a full frame detection mark; If the first state image does not include a full frame detection mark, measuring the distance of the package in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertex of the package area in the material storage frame is located; If the first vertical distance is smaller than a reference vertical distance from the camera device to a horizontal plane where the material storage frame opening is located, it is determined that the material storage frame is in a full frame state.

2. The method for detecting the state of a material storage frame according to claim 1, characterized in that: The detecting whether the first state image includes a full frame detection mark includes: Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance; According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images; It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

3. The method for detecting the state of a material storage frame according to claim 1, characterized in that: The step of measuring the distance of the packages in the material storage frame to determine a first vertical distance between the camera device and a horizontal plane where the vertices of the package area in the material storage frame are located comprises: Get the preset package width range information; Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information; Obtaining predetermined focal length information of a camera device; A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

4. The method for detecting the state of a material storage frame according to claim 3, characterized in that: The calculating, according to the package width range information, basic actual width information of the package in the field of view of the camera device includes: Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range; The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

5. The method for detecting the state of a material storage frame according to claim 1, characterized in that: The method further comprises: If the first state image includes a full frame detection mark, measuring the distance of the package in the material storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located; If the second vertical distance is smaller than the reference vertical distance, acquiring a second state image of the material storage frame opening captured by a camera device; Detecting whether the second state image includes a full frame detection mark; If the second state image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, an abnormal prompt message is sent to the preset user terminal.

6. A sorting system, characterized in that: The sorting system comprises a material storage frame and a visual inspection device, wherein the visual inspection device comprises a camera device and a control device, wherein the camera device is arranged at a preset height above the opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined plane, wherein the angle between the inclined plane and the horizontal line at the bottom of the camera device is an acute angle, and a full frame detection mark is arranged on the inner wall of the inclined plane, and the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located; The camera device is used to collect a first state image of the opening of the material storage frame and transmit the first state image to the control device; The control device is used to receive the first state image captured and transmitted by the camera device; detect whether the first state image includes a full frame detection mark; if the first state image does not include the full frame detection mark, measure the distance of the package in the storage frame to determine the first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than the reference vertical distance from the camera device to the horizontal plane where the storage frame opening is located, determine that the storage frame is in a full frame state.

7. The sorting system according to claim 6, characterized in that: The control device is specifically used for: Determine image segmentation size information according to the material storage frame size or minimum package size information obtained in advance; According to the image segmentation size information, the first state image is segmented into multiple regions to obtain a plurality of segmented images; It is respectively detected whether the plurality of segmented images include a full frame detection mark, and if the plurality of segmented images include a full frame detection mark, it is determined that the first state image includes the full frame detection mark.

8. The sorting system according to claim 6, characterized in that: The control device is specifically used for: Get the preset package width range information; Calculating basic actual width information of the parcel in the field of view of the camera device according to the parcel width range information; Obtaining predetermined focal length information of a camera device; A first vertical distance between the camera device and a horizontal plane where the vertex of the wrapped area in the material storage frame is located is calculated according to the basic actual width information and the focal length information of the camera device.

9. The sorting system according to claim 8, characterized in that: The control device is specifically used for: Calculate the average of the maximum value of the parcel width range and the minimum value of the parcel width range in the parcel width range; The average value is used as the basic actual width of the package in the field of view of the camera device to obtain basic actual width information of the package in the field of view of the camera device.

10. The sorting system according to claim 6, characterized in that: The control device is also used for: If the first state image includes a full frame detection mark, measuring the distance of the package in the material storage frame to determine a second vertical distance between the camera device and the horizontal plane where the vertex of the package area in the material storage frame is located; If the second vertical distance is smaller than the reference vertical distance, acquiring a second state image of the material storage frame opening captured by a camera device; Detecting whether the second state image includes a full frame detection mark; If the second state image includes a full frame detection mark, and the third vertical distance between the current camera device and the horizontal plane where the vertex of the wrapped area in the storage frame is located is still less than the reference vertical distance, an abnormal prompt message is sent to the preset user terminal.

11. A material storage frame state detection device, characterized in that: Applied to a sorting system, the sorting system comprises a material storage frame and a visual detection device, the visual detection device comprises a camera device and a control device, the camera device is arranged at a preset height above the opening of the material storage frame, and the control device is network-connected to the camera device; one side of the material storage frame is an inclined plane, the angle between the inclined plane and the horizontal line at the bottom of the camera device is an acute angle, a full frame detection mark is arranged on the inner wall of the inclined plane, the full frame detection mark is at a preset distance from the horizontal plane where the opening of the material storage frame is located, the material storage frame state detection device is located in the control device, and the device comprises: An image acquisition module, used to acquire a first state image of the material storage frame opening captured by a camera device; A mark detection module, used to detect whether the first state image includes a full frame detection mark; A state detection module is used to measure the distance of the package in the storage frame if the first state image does not include a full frame detection mark, and determine a first vertical distance between the camera device and the horizontal plane where the vertex of the package area in the storage frame is located; if the first vertical distance is less than a reference vertical distance from the camera device to the horizontal plane where the opening of the storage frame is located, it is determined that the storage frame is in a full frame state.

12. A control device, characterized in that: The control device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to implement the material storage frame status detection method according to any one of claims 1 to 5.

13. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the material storage frame status detection method according to any one of claims 1 to 5.

Images