A method and device for cargo stack statistics based on image recognition

Abstract

The present application provides a method for counting cargo stacks based on image recognition, including: obtaining an image of the cargo stack in a first direction and an image in a second direction collected by an image acquisition device; determining a first region of the cargo stack in the image in the first direction and a second region in the image in the second direction based on a trained picture recognition model; calculating the actual size corresponding to the contour line of the first region according to a first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction; and calculating the actual size corresponding to the contour line of the second region according to a second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction; calculating the length, width and height of the cargo stack according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region, so as to achieve automatic supervision, improve the supervision efficiency, reduce the labor cost, and facilitate the intelligent management of the warehouse.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular, to a method and device for cargo stack statistics based on image recognition. Background Art

[0002] A cargo stack refers to the goods stacked in a site. For example, the goods are stacked orderly in a designated area of a warehouse. Currently, as more and more goods are stacked in the site, it is difficult for managers to count the goods, and it may be impossible for managers to tell whether the cargo stack has changed just by the naked eye, which is not conducive to the effective quantitative supervision of the goods.

[0003] The safety of the goods in the warehouse is related to the interests of all parties, so it is of great significance to supervise the goods in the warehouse. Although it is possible to conduct manual inspections or monitor through cameras, it increases the labor cost and is difficult to supervise continuously all day long. Summary of the Invention

[0004] In view of this, to achieve the supervision of the cargo stack, this application provides a method and device for cargo stack statistics based on image recognition.

[0005] Specifically, this application is implemented through the following technical solutions:

[0006] In a first aspect, this application proposes a method for cargo stack statistics based on image recognition, and the method includes:

[0007] Obtain the image of the cargo stack in the first direction and the image of the cargo stack in the second direction collected by the image acquisition device; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0008] Based on the trained picture recognition model, determine the first area of the cargo stack in the image in the first direction and the second area of the cargo stack in the image in the second direction;

[0009] According to the first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, calculate the actual size corresponding to the contour line of the first area; and, according to the second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, calculate the actual size corresponding to the contour line of the second area;

[0010] Calculate the first height, the first length and the first width of the cargo stack according to the actual size corresponding to the contour line of the first area and the actual size corresponding to the contour line of the second area.

[0011] In an embodiment of the present disclosure, the method further includes:

[0012] Calculate the volume of the cargo stack based on the first height, first length, and first width of the cargo stack.

[0013] In one embodiment of the present disclosure, the method further includes:

[0014] Based on the trained image recognition model, determine the type of the goods corresponding to the cargo stack, and calculate the quantity and / or weight of the goods in the cargo stack according to the volume of the cargo stack and the parameter information corresponding to the goods.

[0015] In one embodiment of the present disclosure, the calculating the first height, first length, and first width of the cargo stack according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region includes:

[0016] Determine the second length and second height of the cargo stack according to the actual size corresponding to the contour line of the first region; and determine the second width and third height of the cargo stack according to the actual size corresponding to the contour line of the second region;

[0017] Calculate the first height, first length, and first width of the cargo stack based on the second length, second height, second width, and third height.

[0018] In one embodiment of the present disclosure, the calculating the first height, first length, and first width of the cargo stack based on the second length, second height, second width, and third height includes:

[0019] Calculate the initial value of the corrected height according to the second height and the third height;

[0020] Calculate the corrected length according to the initial value of the corrected height and the proportional relationship between the second length and the second height;

[0021] Calculate the corrected width according to the initial value of the corrected height and the proportional relationship between the second width and the third height;

[0022] Through linear regression, calculate the optimal value of the corrected height that minimizes the sum of the squares of the length error, width error, and height error of the cargo stack; wherein, the length error is the difference between the corrected length and the second length; the width error is the difference between the corrected width and the second width; the height error is the difference between the corrected height and the second height or the third height;

[0023] Take the optimal value of the corrected height as the first height, and calculate the first length and the first width respectively according to the proportional relationship between the second length and the second height, and the proportional relationship between the second width and the third height.

[0024] In one embodiment of the present disclosure, the method further includes:

[0025] Obtain the image in the first direction and the image in the second direction from the video stream of the image acquisition device based on a preset time period, and use image fingerprint technology to determine whether the cargo stack has changed;

[0026] If it is determined that neither the image in the first direction nor the image in the second direction has changed, then perform statistics on the cargo stack.

[0027] In one embodiment of the present disclosure, the method further includes:

[0028] If it is determined that the cargo stack in at least one of the images in the first direction and the image in the second direction has changed, then send a warning message.

[0029] In one embodiment of the present disclosure, the image acquisition device includes an image acquisition device in the first direction and an image acquisition device in the second direction;

[0030] The step of obtaining the image of the cargo stack in the first direction and the image of the cargo stack in the second direction collected by the image acquisition device includes:

[0031] Obtain the image of the cargo stack in the first direction collected by the image acquisition device in the first direction, and obtain the image of the cargo stack in the second direction collected by the image acquisition device in the second direction.

[0032] In one embodiment of the present disclosure, the establishment of the mapping relationship includes:

[0033] Determine the actual length of a preset marking line or marking object in the physical space;

[0034] Determine the pixel length of the marking line or marking object in the image;

[0035] Take the proportional relationship between the pixel length and the actual length of the marking line or marking object as the mapping relationship between the pixel size of the image and the physical space size.

[0036] In a second aspect, the present application proposes a cargo stack statistics device based on image recognition, and the device includes:

[0037] An acquisition unit acquires an image of the cargo stack in a first direction and an image of the cargo stack in a second direction collected by an image acquisition device; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0038] An area determination unit determines a first area in the image of the cargo stack in the first direction and a second area in the image of the cargo stack in the second direction based on a trained picture recognition model;

[0039] A mapping calculation unit calculates the actual size corresponding to the contour line of the first area according to a first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction; and calculates the actual size corresponding to the contour line of the second area according to a second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction;

[0040] A cargo stack calculation unit calculates a first height, a first length, and a first width of the cargo stack according to the actual size corresponding to the contour line of the first area and the actual size corresponding to the contour line of the second area.

[0041] The technical solution provided by the embodiments of the present application may include the following beneficial effects:

[0042] By collecting images through an image acquisition device, identifying the cargo stack area in the images, determining the actual size corresponding to the contour line of the area according to the mapping relationship, and further calculating the length, width, and height of the cargo stack. In the above technical solution, since the images collected from two directions are combined and the two images are associated by using the contour line of the cargo stack, the length, width, and height of the cargo stack can be finally counted, thereby realizing automatic supervision, improving the supervision efficiency, reducing the labor cost, and facilitating the intelligent management of the warehouse.

[0043] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of a method for counting a cargo stack based on image recognition shown in an exemplary embodiment of the present application;

[0045] Figure 2 is a hardware structure diagram of an electronic device shown in an exemplary embodiment of the present application;

[0046] Figure 3 is a block diagram of a device for counting a cargo stack based on image recognition shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0049] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0050] Currently, although the situation in the warehouse can be monitored in real time through a camera, and artificial intelligence can also be used to assist in the monitoring, artificial intelligence mainly realizes the detection and tracking of targets, and is usually used to judge whether there are personnel entering or leaving, or to judge the entry and exit of goods, and cannot realize the quantitative statistics of goods.

[0051] In view of this, the present application provides a technical solution that respectively determines the corresponding regions of the cargo stack in the images in two directions based on a trained image recognition model, and finally calculates the length, width, and height of the cargo stack according to the pre-established mapping relationship to achieve the quantitative statistics of the cargo stack.

[0052] When implementing, an image of the cargo stack in the first direction and an image of the cargo stack in the second direction collected by an image acquisition device can be obtained; wherein, the included angle between the first direction and the second direction reaches a preset angle.

[0053] Then, based on the trained image recognition model, a first region of the cargo stack in the image in the first direction and a second region of the cargo stack in the image in the second direction can be determined.

[0054] Next, according to the first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, the actual size corresponding to the contour line of the first region can be calculated; and according to the second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, the actual size corresponding to the contour line of the second region can be calculated.

[0055] Finally, according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region, the first height, the first length, and the first width of the cargo stack can be calculated.

[0056] In the above technical solution, an image is collected by an image acquisition device, the cargo stack area in the image is recognized, the actual size corresponding to the contour line of the area is determined according to the mapping relationship, and the length, width, and height of the cargo stack are further calculated. Since the images collected from two directions are combined respectively, and the two images are associated by using the contour line of the cargo stack, the length, width, and height of the cargo stack can be finally counted, so as to realize automatic supervision, improve the supervision efficiency, reduce the labor cost, and facilitate the intelligent management of the warehouse.

[0057] Next, the embodiments of the present application will be described in detail.

[0058] Please refer to Figure 1 , Figure 1 which is a flowchart of a cargo stack statistics method based on image recognition shown in an exemplary embodiment of the present application. As Figure 1 shown, it includes the following steps:

[0059] Step 101: Obtain the image of the cargo stack in the first direction collected by the image acquisition device, and the image of the cargo stack in the second direction; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0060] Step 102: Based on the trained picture recognition model, determine the first region of the cargo stack in the image in the first direction and the second region of the cargo stack in the image in the second direction;

[0061] Step 103: According to the first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, calculate the actual size corresponding to the contour line of the first region; and according to the second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, calculate the actual size corresponding to the contour line of the second region;

[0062] Step 104: According to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region, calculate the first height, the first length, and the first width of the cargo stack.

[0063] In this embodiment, it is possible to obtain an image of the cargo stack in a first direction and an image of the cargo stack in a second direction collected by an image acquisition device.

[0064] Wherein, the included angle between the first direction and the second direction reaches a preset angle.

[0065] For example, the cargo stack can be approximately regarded as a cuboid. The first direction can be the direction perpendicular to the front and back of the cuboid, and the second direction can be the direction perpendicular to the left and right of the cuboid. The included angle between the first direction and the second direction is 90 degrees.

[0066] It can be understood that due to the existence of certain direction errors, relative position errors or angle errors in practical applications, there will be a deviation between the actual situation and the theory. However, this does not affect those skilled in the art from implementing this solution according to this specification. Of course, when implementing, those skilled in the art should minimize the errors in all aspects to ultimately improve the statistical accuracy.

[0067] When obtaining the image of the cargo stack in the first direction and the image in the second direction, the same image acquisition device can be used for collection. For example, a drone or a camera installed on a predetermined track can be used; different image acquisition devices can also be used for collection.

[0068] In an illustrated implementation manner, the above-mentioned image acquisition device may include an image acquisition device in the first direction and an image acquisition device in the second direction.

[0069] Furthermore, it is possible to obtain the image of the cargo stack in the first direction collected by the image acquisition device in the first direction and the image of the cargo stack in the second direction collected by the image acquisition device in the second direction.

[0070] For example, a camera can be arranged on the front and side of the cargo stack respectively. The camera at the first position has its lens axis perpendicular to the front of the cargo stack and is responsible for collecting the image of the front of the cargo stack, while the camera at the second position has its lens axis perpendicular to the side of the cargo stack and is responsible for collecting the image of the side of the cargo stack.

[0071] In this embodiment, based on a trained picture recognition model, it is possible to determine a first area in the image of the cargo stack in the first direction and a second area in the image of the cargo stack in the second direction.

[0072] Specifically, based on a trained picture recognition model, it is possible to determine a first area in the collected image of the cargo stack in the first direction and a second area in the collected image of the cargo stack in the second direction.

[0073] Among them, for the above-mentioned trained image recognition model, those skilled in the art can train the machine learning model based on pre-prepared image samples, so that the trained model can output the area of the cargo stack in the input image. Usually, the area of the cargo stack in the image can be framed with a square box, and the coordinates of this area in the image can also be given. Those skilled in the art can set it by themselves according to needs, and this application does not make any limitations in this regard.

[0074] For example, as can be seen from the foregoing, since the lens axis is perpendicular to one surface of the cargo stack, the three-dimensional cargo stack in space is presented as a two-dimensional plane of the front or side in the image. Therefore, the framed shape should theoretically be a rectangle. And because one side of the image in the first direction and the image in the second direction coincide, this coincident side is the height of the cargo stack.

[0075] In this embodiment, the actual size corresponding to the contour line of the first region can be calculated according to the first mapping relationship between the pixel size and the physical space size of the image in the first direction established in advance; and the actual size corresponding to the contour line of the second region can be calculated according to the second mapping relationship between the pixel size and the physical space size of the image in the second direction established in advance.

[0076] Specifically, the actual size corresponding to the contour line of the first region determined above can be calculated according to the first mapping relationship between the pixel size and the physical space size of the image in the first direction established in advance; and the actual size corresponding to the contour line of the second region determined above can be calculated according to the second mapping relationship between the pixel size and the physical space size of the image in the second direction established in advance.

[0077] Since an image is composed of multiple pixel blocks, then according to the area occupied by the cargo stack in the image, the pixel size, that is, the number of pixels, that make up this area can be determined. Further, if the physical space size corresponding to each pixel block in an image is known, the actual size of an area in the image can be calculated.

[0078] For example, assume that in a 100*100 image, the pixel size of the rectangular area where the cargo stack is located is 80*60, and the physical space size corresponding to each pixel block is a*a. Then the actual size corresponding to the rectangular area where the cargo stack is located is 80a*60a.

[0079] In an illustrated implementation manner, the establishment of the mapping relationship includes:

[0080] Determine the actual length of a preset marking line or marker in the physical space; determine the pixel length of the marking line or marker in the image; use the proportional relationship between the pixel length of the marking line or marker and the actual length as the mapping relationship between the pixel size of the image and the size of the physical space.

[0081] It should be noted that the above actual length is pre-measured by relevant personnel. The above marking line can be a line segment pre-drawn by those skilled in the art within the visible range of the image acquisition device, or it can be the boundary line of the goods storage area where the goods stack is located. The above marker can be the base or tray for preventing goods. The present application does not make any limitations in this regard.

[0082] When determining the pixel length of the marking line or marker in the image, it can be calculated according to the coordinates of the marking line or marker in the image, or the starting point and ending point of the marking line or marker can be marked on the image, and the number of pixel blocks can be calculated to obtain it.

[0083] For example, if the length of the marking line is 10m and the pixel length of the marking line in the image is 100 pixel blocks, then the actual length corresponding to each pixel block is 10cm, that is, the proportional relationship between the pixel length and the actual length is 1:10cm.

[0084] In addition, since the above mapping relationship is different when the image acquisition device is at different distances from the goods stack, therefore, after the above mapping relationship is established at a certain position of the image acquisition device, unless it can be determined that the image acquisition device can still be located at that position, otherwise the position of the image acquisition device should not be changed. If there is a change and it cannot be reset to the original position, those skilled in the art should re-establish the above mapping relationship.

[0085] In this embodiment, the first height, first length, and first width of the goods stack can be calculated according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region.

[0086] Specifically, the first height, first length, and first width of the goods stack can be calculated according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region calculated above.

[0087] For example, assume that the actual size corresponding to the contour line of the first region is a*b1, and the actual size corresponding to the contour line of the second region is c*b2. As can be seen from the foregoing, since there is an overlapping part between the contour line of the first region and the contour line of the second region, this part can be used as the height of the goods stack. Then, assuming the method of taking the average value is used, the corresponding length, width, and height of the goods stack are a, c, and (b1 + b2) / 2 respectively.

[0088] In one of the illustrated embodiments, the volume of the cargo stack can be calculated based on the first height, the first length, and the first width of the cargo stack.

[0089] Based on the first height, the first length, and the first width of the cargo stack determined above, the volume of the cargo stack can be calculated according to the volume formula.

[0090] For example, assuming that the first height, the first length, and the first width of the above-mentioned cargo stack are a, b, and c respectively, then since the cargo stack should theoretically be a cuboid, the volume of the cargo stack is a * b * c.

[0091] In another illustrated embodiment, based on the trained image recognition model, the type of the goods corresponding to the cargo stack can be determined, and according to the volume of the cargo stack and the parameter information corresponding to the goods, the quantity and / or the weight of the goods in the cargo stack can be calculated.

[0092] It should be noted that since there are usually multiple storage areas in the warehouse and the types of goods corresponding to each storage area may be different, the type of the goods corresponding to the cargo stack can be determined based on the trained image recognition model, and the parameter information of the goods can be obtained according to the type of the goods. For example, the volume of each good, or the volume of the cuboid space occupied by each good, or the weight of each good is pre-measured, and this information is used as the parameter information of the goods.

[0093] Furthermore, in the case where the volume of the cargo stack is calculated, the quantity of the goods can be calculated according to the volume of each good, and then the weight of the cargo stack can be calculated according to the weight of each good.

[0094] For example, assuming that the volume of the cargo stack is N and the volume of each good is M, since the cargo stack is theoretically a cuboid and the goods are also cuboids and are arranged according to rules, the quantity of the goods is N / M; furthermore, if the weight of each good is W, then the weight of the cargo stack is WN / M.

[0095] In one of the illustrated embodiments, before performing the cargo stack statistics, the images in the first direction and the images in the second direction can also be obtained from the video stream of the image acquisition device based on a preset time period, and the image fingerprint technology can be used to determine whether the cargo stack has changed; if it is determined that neither the images in the first direction nor the images in the second direction have changed, then the cargo stack is statistically analyzed.

[0096] Specifically, an image acquisition device can be deployed for real-time monitoring. Based on a preset time period, frames are regularly extracted from the video stream to obtain multiple frames of pictures in the first direction and the second direction respectively, and the image fingerprint technology is used to determine the similarity between multiple frames of pictures in each direction, so as to determine whether the cargo stack in the image has changed.

[0097] Among them, the image fingerprint technology refers to generating a "fingerprint" string for each image, and then comparing the fingerprints of different images. The closer the results are, the more similar the images are.

[0098] In addition, before making a judgment, the images obtained by frame extraction can be preprocessed first. For example, pictures with too dark or too bright light can be excluded.

[0099] Furthermore, if the images in the first direction and the second direction have not changed, it can be considered that the cargo stack has not changed, and the statistical operation on the cargo stack can be continued.

[0100] In another illustrated embodiment, if it is determined that the cargo stack in at least one of the images in the first direction and the second direction has changed, a warning message is sent.

[0101] It can be understood that if the image in the first direction has changed, or the image in the second direction has changed, or both have changed, it can be considered that the cargo stack has changed and a warning is required, and a warning message is sent to the relevant personnel.

[0102] In an illustrated embodiment, the second length and the second height of the cargo stack can be determined according to the actual size corresponding to the contour line of the first region; and the second width and the third height of the cargo stack can be determined according to the actual size corresponding to the contour line of the second region; based on the second length, the second height, the second width, and the third height, the first height, the first length, and the first width of the cargo stack are calculated.

[0103] It should be noted that since the above regions should theoretically be rectangles, those skilled in the art can set the deviation thresholds for the upper and lower contour lines and the left and right contour lines of the regions when implementing. If the deviation is less than the threshold, the average value method can be used to determine the size of the contour line. If the deviation is greater than the threshold, the position of the image acquisition device may need to be adjusted. This application does not make any limitations in this regard.

[0104] In an example, the second length can be used as the first length of the cargo stack, the second width can be used as the first width of the cargo stack, and the average value of the second height and the third height can be used as the first height.

[0105] In order to obtain more accurate dimensions, an algorithm can be used to correct the dimensions of the cargo stack.

[0106] In an illustrated embodiment, the initial value of the corrected height can be calculated according to the second height and the third height;

[0107] Calculate the corrected length according to the initial value of the corrected height and the proportional relationship between the second length and the second height;

[0108] Calculate the corrected width according to the initial value of the corrected height and the proportional relationship between the second width and the third height;

[0109] Through linear regression, calculate the optimal value of the corrected height that minimizes the sum of the squares of the length error, width error, and height error of the cargo stack; wherein, the length error is the difference between the corrected length and the second length; the width error is the difference between the corrected width and the second width; the height error is the difference between the corrected height and the second height or the third height;

[0110] Take the optimal value of the corrected height as the first height, and calculate the first length and the first width respectively according to the proportional relationship between the second length and the second height, and the proportional relationship between the second width and the third height.

[0111] For example, assume that the second height is H2 and the third height is H3, then an initial value Ha of the corrected height can be determined, and the value of Ha can be between H2 - H3.

[0112] Then, the corrected length La = (second length L2 / second height H2) * corrected height Ha, and the corrected width Wa = (second width W2 / third height H3) * corrected height Ha.

[0113] Through linear regression, calculate the optimal value of the corrected height that minimizes the value of (La - L2) 2 +(Wa - W2) 2 +(Ha - H2) 2 Assume that the optimal value is Hb.

[0114] Then the first height of the cargo stack is Hb, the first length is L2 * Hb / H2, and the first width is W2 * Hb / H3.

[0115] In the above technical solution, images are collected by an image acquisition device, the cargo stack area in the images is recognized, the actual sizes corresponding to the contour lines of the area are determined according to the mapping relationship, and the length, width, and height of the cargo stack are further calculated. Since the images collected from two directions are combined, the two images are associated using the contour line of the cargo stack, and finally the length, width, and height of the cargo stack can be counted, thereby realizing automatic supervision, improving the supervision efficiency, reducing the labor cost, and facilitating the intelligent management of the warehouse.

[0116] Corresponding to the above method embodiment, the present application also provides an embodiment of a device.

[0117] Corresponding to the above method embodiments, the present application also provides an embodiment of a cargo stack statistics device based on image recognition. The embodiment of the cargo stack statistics device based on image recognition of the present application can be applied to an electronic device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of the electronic device where it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. At the hardware level, as Figure 2 shown, it is a hardware structure diagram of an electronic device shown in an exemplary embodiment of the present application. In addition to Figure 2 the shown processor, memory, network interface, and non-volatile memory, the electronic device where the device is located in the embodiment usually further includes other hardware according to the actual functions of the electronic device, which will not be elaborated here.

[0118] Please refer to Figure 3 , Figure 3 which is a block diagram of a cargo stack statistics device based on image recognition shown in an exemplary embodiment of the present application. As Figure 3 shown, the cargo stack statistics device 300 based on image recognition can be applied to the aforementioned Figure 2 shown electronic device, and includes:

[0119] An acquisition unit 301, which acquires the image of the cargo stack in the first direction and the image of the cargo stack in the second direction collected by the image acquisition device; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0120] A region determination unit 302, which determines the first region in the image of the cargo stack in the first direction and the second region in the image of the cargo stack in the second direction based on the trained picture recognition model;

[0121] A mapping calculation unit 303, which calculates the actual size corresponding to the contour line of the first region according to the first mapping relationship between the pixel size and the physical space size of the image in the first direction established in advance; and calculates the actual size corresponding to the contour line of the second region according to the second mapping relationship between the pixel size and the physical space size of the image in the second direction established in advance;

[0122] A cargo stack calculation unit 304, which calculates the first height, the first length, and the first width of the cargo stack according to the actual size corresponding to the contour line of the first region and the actual size corresponding to the contour line of the second region.

[0123] In one embodiment, the device 300 further includes:

[0124] A volume calculation unit calculates the volume of the cargo stack based on the first height, the first length, and the first width of the cargo stack.

[0125] In one embodiment, the device 300 further includes:

[0126] A refinement calculation unit determines the type of the cargo corresponding to the cargo stack based on the trained picture recognition model, and calculates the quantity and / or the weight of the cargo in the cargo stack according to the volume of the cargo stack and the parameter information corresponding to the cargo.

[0127] In one embodiment, the cargo stack calculation unit 304 further:

[0128] Determines the second length and the second height of the cargo stack according to the actual dimensions corresponding to the contour line of the first region; and determines the second width and the third height of the cargo stack according to the actual dimensions corresponding to the contour line of the second region;

[0129] Calculates the first height, the first length, and the first width of the cargo stack based on the second length, the second height, the second width, and the third height.

[0130] In one embodiment, the cargo stack calculation unit 304 further:

[0131] Calculates an initial value of the correction height according to the second height and the third height;

[0132] Calculates the correction length according to the initial value of the correction height and the proportional relationship between the second length and the second height;

[0133] Calculates the correction width according to the initial value of the correction height and the proportional relationship between the second width and the third height;

[0134] Calculates an optimal value of the correction height that minimizes the sum of the squares of the length error, the width error, and the height error of the cargo stack through linear regression; wherein, the length error is the difference between the correction length and the second length; the width error is the difference between the correction width and the second width; the height error is the difference between the correction height and the second height or the third height;

[0135] Uses the optimal value of the correction height as the first height, and calculates the first length and the first width respectively according to the proportional relationship between the second length and the second height, and the proportional relationship between the second width and the third height.

[0136] In one embodiment, the device 300 further includes:

[0137] A judgment unit obtains the images in the first direction and the images in the second direction from the video stream of the image acquisition device based on a preset time period, and uses image fingerprint technology to judge whether the cargo stack has changed; if it is determined that neither the images in the first direction nor the images in the second direction have changed, then statistics are performed on the cargo stack.

[0138] In one embodiment, the apparatus 300 further includes:

[0139] A warning unit sends a warning message if it is determined that the cargo stack in at least one of the images in the first direction and the images in the second direction has changed.

[0140] In one embodiment, the image acquisition device includes an image acquisition device in the first direction and an image acquisition device in the second direction;

[0141] The obtaining unit 301 further:

[0142] Obtains the image of the cargo stack in the first direction collected by the image acquisition device in the first direction, and obtains the image of the cargo stack in the second direction collected by the image acquisition device in the second direction.

[0143] In one embodiment, the mapping calculation unit 303 further:

[0144] Determines the actual length of a preset marking line or marking object in the physical space;

[0145] Determines the pixel length of the marking line or marking object in the image;

[0146] Uses the proportional relationship between the pixel length and the actual length of the marking line or marking object as the mapping relationship between the pixel size of the image and the size of the physical space.

[0147] Each embodiment in this application is described in a progressive manner. For the same / similar parts between each embodiment, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the client device embodiment and the apparatus embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment.

[0148] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0149] The devices, devices, modules or modules illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0150] Corresponding to the above method embodiments, this specification also provides an embodiment of an electronic device. The electronic device includes: a processor and a memory for storing machine-executable instructions; wherein, the processor and the memory are usually connected to each other through an internal bus. In other possible implementation manners, the device may further include an external interface to be able to communicate with other devices or components.

[0151] In this embodiment, by reading and executing the machine-executable instructions stored in the memory corresponding to the user authentication logic, the processor is caused to:

[0152] Obtain images of the cargo stack in a first direction and images of the cargo stack in a second direction collected by an image acquisition device; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0153] Based on a trained picture recognition model, determine a first area in the image of the cargo stack in the first direction and a second area in the image of the cargo stack in the second direction;

[0154] According to a first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, calculate the actual size corresponding to the contour line of the first area; and, according to a second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, calculate the actual size corresponding to the contour line of the second area;

[0155] Calculate the first height, first length, and first width of the cargo stack according to the actual sizes corresponding to the contour lines of the first region and the actual sizes corresponding to the contour lines of the second region.

[0156] Corresponding to the above method embodiments, this specification also provides an embodiment of another electronic device. The electronic device includes: a processor and a memory for storing machine-executable instructions; wherein, the processor and the memory are generally interconnected through an internal bus. In other possible implementation manners, the device may further include an external interface to be able to communicate with other devices or components.

[0157] In this embodiment, by reading and executing the machine-executable instructions stored in the memory corresponding to the user authentication logic, the processor is caused to:

[0158] Obtain the image of the cargo stack in the first direction collected by the image acquisition device, and the image of the cargo stack in the second direction; wherein, the included angle between the first direction and the second direction reaches a preset angle;

[0159] Based on the trained picture recognition model, determine the first region in the image of the cargo stack in the first direction and the second region in the image of the cargo stack in the second direction;

[0160] According to the first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, calculate the actual size corresponding to the contour line of the first region; and, according to the second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, calculate the actual size corresponding to the contour line of the second region;

[0161] Calculate the first height, first length, and first width of the cargo stack according to the actual sizes corresponding to the contour lines of the first region and the actual sizes corresponding to the contour lines of the second region.

[0162] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation manners of this application. This application is intended to cover any variations, uses, or adaptive changes of this application, and these variations, uses, or adaptive changes follow the general principles of this application and include the common general knowledge or conventional technical means in the technical field not disclosed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0163] It should be understood that this application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

[0164] The foregoing are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for counting stacks of goods based on image recognition, the method comprises: Obtaining images of the stack of goods in a first direction and images of the stack of goods in a second direction collected by an image acquisition device; wherein, the included angle between the first direction and the second direction reaches a preset angle; Based on a trained picture recognition model, determining a first area of the stack of goods in the image in the first direction and a second area of the stack of goods in the image in the second direction; According to a first mapping relationship established in advance between the pixel size and the physical space size of the image in the first direction, calculating the actual size corresponding to the contour line of the first area; and, according to a second mapping relationship established in advance between the pixel size and the physical space size of the image in the second direction, calculating the actual size corresponding to the contour line of the second area; According to the actual size corresponding to the contour line of the first area, determining a second length and a second height of the stack of goods; and, according to the actual size corresponding to the contour line of the second area, determining a second width and a third height of the stack of goods; Based on the second length, the second height, the second width and the third height, calculating a first height, a first length and a first width of the stack of goods.

2. The method according to claim 1, the method further comprises: Based on the first height, the first length and the first width of the stack of goods, calculating the volume of the stack of goods.

3. The method according to claim 2, the method further comprises: Based on the trained picture recognition model, determining the type of goods corresponding to the stack of goods, and according to the volume of the stack of goods and the parameter information corresponding to the goods, calculating the quantity of goods in the stack of goods and / or the weight of the stack of goods.

4. The method according to claim 1, the calculating the first height, the first length and the first width of the stack of goods based on the second length, the second height, the second width and the third height, comprises: Calculating an initial value of the corrected height according to the second height and the third height; Calculating the corrected length according to the initial value of the corrected height and the proportional relationship between the second length and the second height; Calculating the corrected width according to the initial value of the corrected height and the proportional relationship between the second width and the third height; Through linear regression, calculating an optimal value of the corrected height that minimizes the sum of squares of the length error, width error and height error of the stack of goods; wherein, the length error is the difference between the corrected length and the second length; the width error is the difference between the corrected width and the second width; the height error is the difference between the corrected height and the second height or the third height; Taking the optimal value of the corrected height as the first height, and respectively calculating the first length and the first width according to the proportional relationship between the second length and the second height and the proportional relationship between the second width and the third height.

5. The method according to claim 1, the method further comprises: Obtain the images in the first direction and the images in the second direction from the video stream of the image acquisition device based on a preset time period, and use image fingerprint technology to determine whether the cargo stack has changed; If it is determined that neither the images in the first direction nor the images in the second direction have changed, then perform statistics on the cargo stack.

6. The method according to claim 5, the method further includes: If it is determined that the cargo stack in at least one of the images in the first direction and the images in the second direction has changed, then send a warning message.

7. The method according to claim 1, the image acquisition device includes an image acquisition device in the first direction and an image acquisition device in the second direction; The obtaining of the image in the first direction of the cargo stack collected by the image acquisition device, and the image in the second direction of the cargo stack, includes: Obtain the image in the first direction of the cargo stack collected by the image acquisition device in the first direction, and obtain the image in the second direction of the cargo stack collected by the image acquisition device in the second direction.

8. The establishment of the mapping relationship according to claim 1, includes: Determine the actual length of a preset marking line or marking object in physical space; Determine the pixel length of the marking line or marking object in the image; Take the proportional relationship between the pixel length of the marking line or marking object and the actual length as the mapping relationship between the pixel size of the image and the physical space size.

9. A cargo stack statistics device based on image recognition, the device includes: An acquisition unit that acquires the image in the first direction of the cargo stack collected by the image acquisition device, and the image in the second direction of the cargo stack; wherein, the included angle between the first direction and the second direction reaches a preset angle; A region determination unit that determines a first region of the cargo stack in the image in the first direction and a second region of the cargo stack in the image in the second direction based on a trained picture recognition model; A mapping calculation unit that calculates the actual size corresponding to the contour line of the first region according to a first mapping relationship between the pixel size and the physical space size of the image in the first direction established in advance; and calculates the actual size corresponding to the contour line of the second region according to a second mapping relationship between the pixel size and the physical space size of the image in the second direction established in advance; A cargo stack calculation unit that determines a second length and a second height of the cargo stack according to the actual size corresponding to the contour line of the first region; and determines a second width and a third height of the cargo stack according to the actual size corresponding to the contour line of the second region; based on the second length, the second height, the second width and the third height, calculate the first height, the first length and the first width of the cargo stack.

Images