Counting device and counting method

Abstract

To provide a counting device that can easily count materials and other items without using special counting aids. [Solution] A counting device 200 for counting the number of plate-shaped materials stacked in a predetermined direction, comprising: an image acquisition unit 110 that acquires two-dimensional image information from a mobile terminal; a count target material registration unit 220 that registers material areas in advance using the two-dimensional image information; a count target area recognition unit 230 that recognizes the target area of ​​the materials to be counted from the two-dimensional image information; a distance information extraction unit 260 that extracts distance information between the target area and the mobile terminal based on the two-dimensional image information; a boundary line detection unit 240 that detects the boundary lines of the materials within the target area; and a result correction unit 270 that determines whether the boundary lines are not detected or are falsely detected based on the extracted distance information and the interval of the detected boundary lines, and corrects the detection results of the boundary lines of the materials.

Description

Technical Field

[0001] The present invention relates to a counting device and a counting method.

Background Art

[0002] With the decrease in the number of workers due to the recent declining birthrate and aging population, in order to replace visual confirmation work such as checking the inventory of materials, technologies for counting the number of objects of various materials have been proposed. For example, in Patent Document 1 below, it is disclosed that "a sheet counting device is arranged on the side of a plurality of sheet-like or film-like measurement objects stacked in the thickness direction, and an imaging device that acquires an image of each boundary portion between the stacked measurement objects, and an arithmetic processing device that calculates the number of measurement objects by adding the density data of the boundary portions based on the output of the imaging device, and a counting auxiliary jig that is arranged on at least one of the upper end portion and the lower end portion of the stacked measurement objects and has a counting auxiliary surface that is brought into close contact with the upper end surface of the measurement object from above or the lower end surface of the measurement object from below in the vicinity of the surface for acquiring an image and is larger than the thickness of one measurement object".

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, the number of bundles stacked in a dedicated device is counted in the first place, assuming that the distance between the imaging object and the camera is known and the height of the object in the camera is constant. Therefore, when an object is photographed with a mobile camera (mobile terminal, portable terminal), there is a problem that the number cannot be counted. <0——000026>

[0005] Furthermore, Patent Document 1 assumes that a line CCD camera is used to capture the boundary lines of the material bundles from the front, and does not take the camera's field of view into consideration. Therefore, when taking pictures with a field of view, the spacing of the material boundaries is not constant, and if the object is photographed with a mobile camera as is, the camera's field of view must be taken into consideration.

[0006] The present invention was made to solve the aforementioned problems, and aims to provide a counting device and counting method that can easily count materials and the like without using special counting aids. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a counting device for counting the number of plate-shaped materials stacked in a predetermined direction, comprising: an image acquisition unit that acquires two-dimensional image information from a mobile terminal; a target material registration unit that registers material areas in advance using the two-dimensional image information; a target area recognition unit that recognizes the target area of ​​the material to be counted from the two-dimensional image information; and based on the two-dimensional image information, Based on an AI learning model that considers the relationship between the distance between the target area and the mobile terminal and the monocular depth estimation of the 2D image, The present invention is characterized by comprising: a distance information extraction unit that extracts distance information between the target area and the mobile terminal; a boundary detection unit that detects the boundary lines of the material within the target area; and a result correction unit that determines whether the boundary lines are not detected or are falsely detected based on the distance between the extracted distance information and the detected boundary lines, and corrects the detection result of the boundary lines of the material. Other aspects of the present invention will be described in the embodiments described below. [Effects of the Invention]

[0008] According to the present invention, materials and the like can be easily counted without using special counting aids. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of a counting device according to an embodiment. [Figure 2] This is a block diagram of the computer configuration. [Figure 3]This flowchart shows the pre-registration process for materials to be counted. [Figure 4] This is an explanatory diagram showing an example of pre-registration of materials to be counted. [Figure 5] This is a flowchart showing the material counting process during material counting. [Figure 6] This flowchart shows the details of the process for extracting material boundaries from images of material bundles. [Figure 7] This is an explanatory diagram illustrating an example of the process for extracting material boundaries from an image of a material bundle. [Figure 8] This is an explanatory diagram illustrating an example of a method for correcting material boundary lines. [Figure 9] This is an explanatory diagram showing examples of material boundary lines that are not detected or are falsely detected. [Figure 10] This is an explanatory diagram showing an example of detecting a characteristic area on the front surface of a material. [Figure 11] This is a diagram showing an example of a scene where bundles of materials are being photographed. [Figure 12A] This figure shows an example of the screen displayed when selecting counting materials on a mobile terminal and pressing the capture button. [Figure 12B] This figure shows an example of the screen displayed when uploading an image from a mobile device and when the count results are shown. [Figure 12C] This figure shows an example of the screen displayed when selecting an image count result on a mobile terminal and when displaying the detailed count result. [Figure 13] This diagram shows an overview of the software framework. [Modes for carrying out the invention]

[0010] [Summary of the Embodiment] The counting device of the embodiment can easily count materials and the like without using a special counting aid by using an image of a bundle of materials, particularly a two-dimensional image, taken by a mobile camera (mobile terminal, mobile phone, etc.). The counting device can count the number of sheets of materials in a bundle of materials in which a plurality of plate-like materials are stacked in the vertical direction. Note that the stacking in the vertical direction in the embodiment is an example of materials "stacked in a predetermined direction", and the present disclosure can also be applied to counting the number of materials in a bundle of materials in which plate-like materials (sheet-like materials) are stacked in the horizontal direction.

[0011] FIG. 11 is a diagram showing an example of a photographed scene of a material bundle 370. In FIG. 11, a bundle of materials 370 in which a plurality of plate-like materials are stacked in the vertical direction is stacked to about the height of a person. The materials shown in FIG. 11 are those having a characteristic region (see FIG. 10) on the front surface of the material, and the handling of this characteristic region will be described later. The material bundle 370 is photographed by an image acquisition device 100 which is a mobile camera, and the number of materials can be counted by a counting device based on the photographed image. It is characteristic that even if the photographer takes a relatively rough (flexible) shot with respect to the angle of view by the mobile camera, there is no problem in calculating the number of materials. Note that the angle of view is the range of the area captured in the photograph when shooting with a camera, expressed in terms of an angle.

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a counting device 200 according to an embodiment. The counting device 200 is a web server and is a counting device that counts the number of sheets of the material in which a plurality of plate-like materials are stacked in the vertical direction based on two-dimensional image information from an image acquisition device 100 (mobile terminal). The counting device 200 is connected to the image acquisition device 100 via a communication network. The image acquisition device 100 is a mobile terminal with a camera such as a smartphone.

[0013] In FIG. 1, the processing at the time of pre-registration of count target materials (material registration) for registering various materials (the upper part of FIG. 1) and the processing at the time of counting materials in different material bundles 370 (the lower part of FIG. 1) are separately described. In the present embodiment, even if the distance between the image acquisition device 100 and the material bundle 370 is different at the time of pre-registration of count target materials and at the time of material counting, appropriate counting can be performed. Here, the outline of the software framework will be described with reference to FIG. 13.

[0014] FIG. 13 is a diagram showing an outline of the software framework. The process of counting the number of materials in the material bundle 370 of the counting device 200 adopts a three-layer system. In the three-layer system, the entire system is divided into three layers: a P layer (presentation layer), an F layer (function layer), and a D layer (data layer), and is implemented on the Web server side. By separating and arranging the processing in multiple layers, it is possible to flexibly respond when a change needs to be made to a certain layer.

[0015] The P layer performs request processing from the browser of the image acquisition device 100 and response processing of the result. It processes a request from the browser and calls the logic of the F layer corresponding to the request (event). Also, it responds to the result from the F layer to the browser.

[0016] The F layer uses classes that are not Web-dependent and can be executed from outside the P layer as a library. It is designed to be a single logic for requests from the browser. The D layer sets the parameters received from the F layer in SQL, executes them, and then returns the result to the F layer.

[0017] On the image acquisition device 100 side, it is advisable to download a software application that supports user-side processes such as the process of taking a picture with the camera of the image acquisition device 100 according to a shooting instruction on the Web screen and the process of extracting the corresponding photo from the photo folder of the image acquisition device 100 from the counting device 200 side.

[0018] Returning to Figure 1, the image acquisition device 100 includes an image acquisition unit 110, a material position selection unit 120, a data transmission unit 130, a material selection unit 140, a count result display unit 150, and the like. The counting device 200 includes a data receiving unit 210, a material registration unit 220 (target material registration unit), material information 300, a region recognition unit 230 (target region recognition unit), a boundary line detection unit 240, a feature region detection unit 250, a distance information extraction unit 260, a result correction unit 270, a counting result registration unit 280, and count result information 310.

[0019] This section provides an overview of the functions of each part. During the pre-registration of countable materials, the image acquisition unit 110 acquires two-dimensional image information from the image acquisition device 100. The material position selection unit 120 allows the user to pre-select the position of the material area in the two-dimensional image information. The countable material registration unit 220 (target material registration unit) pre-registers the material areas within the material bundle 370 in the countable material information 300 based on the two-dimensional image information. The countable material registration unit 220 includes a distance estimation processing unit 221 and a material height information extraction processing unit 222, which will be described later. The countable material information 300 includes the material name, material type, material height information, distance information at the time of shooting, etc. of the registered material.

[0020] During material counting, the image acquisition unit 110 acquires two-dimensional image information from the image acquisition device 100. The material selection unit 140, during material counting, allows the user to select the material information to be counted from the pre-registered material information 300. The target area recognition unit 230 (target area recognition unit) recognizes the target area of ​​the material to be counted from the two-dimensional image information. The boundary line detection unit 240 extracts the boundary lines of the material within the target area. The feature area detection unit 250 extracts feature areas on the surface of the material within the target area.

[0021] In Figure 7, which will be described later, the "target area" is the image located in the center of the upper section. In other words, the area of ​​the material bundle extracted from the input image is the "target area." Furthermore, a "boundary line" is the line formed by the top surface of the material below and the bottom surface of the material above it in a stack of materials. In other words, a boundary line is the line on the image that separates the material above from the material below. Specifically, for the top layer of materials in a stack, the boundary line is the line forming its top surface, and for the bottom layer of materials, the boundary line is the line forming its bottom surface.

[0022] The distance information extraction unit 260 extracts distance information between the target area and the mobile terminal based on the two-dimensional image information. The result correction unit 270 determines whether the boundary line is undetected or falsely detected based on the extracted distance information and the detected boundary line interval, and corrects the detection result of the material boundary line. The result correction unit 270 may also correct the detection result of the material boundary line including the feature area. The count result registration unit 280 then registers the detection result corrected by the result correction unit 270 in the count result information 310. The count result display unit 150 displays the number of materials based on the count result information 310. Here, the boundary line interval is basically the thickness of one material (height per material). Details will be described later with reference to Figures 6 and 7.

[0023] Figure 2 is a block diagram of the Computer 980. The image acquisition device 100 and counting device 200 shown in Figure 1 each include one or more computers 980 as shown in Figure 2. In Figure 2, the computer 980 includes a CPU 981 (processing unit), a storage unit 982, a communication interface 983, an input / output interface 984, and a media interface 985. Here, the storage unit 982 includes a RAM 982a, a ROM 982b, and an HDD 982c. The communication interface 983 is connected to the communication circuit 986. The input / output interface 984 is connected to the input / output device 987. The media interface 985 reads and writes data to and from the recording medium 988.

[0024] ROM982b stores the IPL (Initial Program Loader) and other programs executed by the CPU. HDD982c stores application programs and various data. CPU981 implements various functions by executing application programs and other data read from HDD982c into RAM982a. The internal workings of the image acquisition device 100 and counting device 200 shown in Figure 1 above represent the functions implemented by application programs and other data as blocks.

[0025] (When materials to be counted are pre-registered) Figure 3 is a flowchart of the pre-registration process S120 for materials to be counted. Figure 4 is an explanatory diagram showing an example of pre-registration for materials to be counted. The image acquisition unit 110 acquires (takes) an image of the newly registered material bundle 370 (step S100). The material position selection unit 120 draws a box around the area of ​​the material within the acquired material bundle 370 on the image, according to the user's selection (step S101). Then, the data transmission unit 130 transmits the image and rectangular coordinate information, as well as the material name and material type specified by the user, to the counting device 200 (step S102).

[0026] Figure 4 shows the imaging status of the material bundle 370 of the target material by the image acquisition device 100 and the method of specifying the material area 320. In step S101, the user outlines the material area 320 from the image (2D image) of the material bundle 370. In addition to outlining the material, it is also possible to specify two points, the top and bottom edges of the material area 320. Alternatively, multiple materials may be selected and registered along with the number of selected materials.

[0027] Returning to Figure 3, the data receiving unit 210 of the counting device 200 receives data (step S103). The distance estimation processing unit 221 estimates the depth map of the received image data and estimates the distance from the image acquisition device 100 to the material bundle 370 (step S104). The material height information extraction processing unit 222 extracts the height of each material using the rectangular coordinate information (step S105). Then, the count target material registration unit 220 registers the distance from the image acquisition device 100 to the material bundle 370 and the height of each material obtained in steps S104 and S105 in the count target material information 300, associating them with the material name and material type (step S106).

[0028] In step S104, the distance from the image acquisition device 100 to the material bundle 370 can be estimated using a monocular depth estimation technique for 2D images and a model trained by AI on the relationship between the distance from the image acquisition device 100 to the material bundle 370 and the depth estimation technique.

[0029] In step S105, the height of each material is extracted by the number of pixels. Alternatively, the height of each material may be calculated using the information from the depth map, for example, in units such as centimeters.

[0030] (When counting materials) Figure 5 is a flowchart showing the material counting process S220 during material counting. The image acquisition unit 110 acquires (takes) an image of the bundle of materials 370 to be counted (step S200). The material selection unit 140 acquires a list of material information to be counted 300 (step S201), and the user selects the material acquired in step S200 from the list (step S202). The data transmission unit 130 transmits the acquired image and the data of the selected material information to be counted to the counting device 200 (step S203).

[0031] The data receiving unit 210 of the counting device 200 receives data (step S204). The count target area recognition unit 230 extracts the area of ​​the material bundle 370 from the image (step S205). Then, the boundary line detection unit 240 extracts only the material boundary line from the image of the material bundle 370 (step S206, material boundary line extraction process, details will be described later in Figure 6), the feature area detection unit 250 extracts only the features of the material front from the image of the material bundle 370 (step S207), and the distance information extraction unit 260 estimates the distance from the image acquisition device 100 (mobile camera) to the material bundle 370 from the image of the material bundle 370 (step S208).

[0032] The result correction unit 270 corrects the result of step S206 using the processing results of steps S207 and S208 (step S209, result correction processing, details will be described later in Figure 7). The count result registration unit 280 registers the corrected result in the count result information 310 (step S210). The count result display unit 150 displays the count result on the display unit of the image acquisition device 100 based on the count result information 310 (step S211).

[0033] In step S203, the counting material information 300 may send pre-registered information about the selected material (distance to the material bundle and height per material). Alternatively, the counting device 200 may send an index number that can identify the selected material so that it can directly search for the counting material information 300.

[0034] In step S207, feature extraction can be performed by learning with AI and extracting the relevant regions. Alternatively, extraction can be performed using methods such as pattern matching.

[0035] In step S208, similar to step S104, the distance from the image acquisition device 100 to the material bundle 370 can be estimated using a monocular depth estimation technique for 2D images and a model that has been AI-learned to determine the relationship between the distance from the image acquisition device 100 to the material bundle 370 and the depth estimation technique.

[0036] (Extraction process for material boundaries) Figure 6 is a flowchart detailing the process of extracting material boundaries from an image of a material bundle (step S206). Figure 7 is an explanatory diagram showing an example of the process of extracting material boundaries from an image of a material bundle. The process in Figure 6 will be explained with reference to the explanatory diagram in Figure 7.

[0037] The boundary detection unit 240 generates a line drawing from the image and extracts edge information (step S300, see step S300 in Figure 7). The boundary detection unit 240 extracts only the straight line components from the line drawing (step S301), finds the vector of each extracted straight line, and filters it by the length and slope of the vector (step S302). In steps S301 and S302 in Figure 7, the green (white in this figure) lines are the straight lines that remain after filtering (i.e., these lines are the boundary lines).

[0038] To elaborate, as mentioned above, a "boundary line" is the line formed by the top surface of the material below and the bottom surface of the material above it in a stack of materials. In other words, a boundary line is a line in an image that separates the material above from the material below. Specifically, for the top layer of materials in a stack, the boundary line is the line forming its top surface, and for the bottom layer of materials, the boundary line is the line forming its bottom surface. In this disclosure, the boundary line is detected (extracted) by image processing.

[0039] Subsequently, the boundary detection unit 240 counts the number of pixels in the X-shape (horizontal direction) using the search window 330, which contains several pixels in the Y-axis direction (vertical direction) (step S303). In step S303 of Figure 7, the search window 330 is searched sequentially from top to bottom in the Y-axis direction.

[0040] The boundary detection unit 240 graphs the number of counted pixels and extracts the peak (step S304). In step S304 of Figure 7, the vertical axis is the Y-axis direction, the horizontal axis is the number of pixels counted in the search window 330, and the × marks are the peak values. If the peak value is abnormally small, it may be due to external factors such as shadows, or the boundary line may not have been detected due to overexposure, etc. The boundary detection unit 240 then creates a list of Y coordinates where peaks appear (list of material boundaries) (step S305).

[0041] In other words, the boundary detection unit 240 of the counting device 200 performs a filtering process to remove unnecessary straight lines other than boundary lines included in the two-dimensional image information (step S302), obtains the number of pixels of a horizontal straight line from the top edge of the image using the search window 330 for the image after the filtering process (step S303), calculates a peak value based on the amount of change in the number of pixels (step S304), and performs a peak calculation process to extract and list the coordinates corresponding to the peak value as the boundary line of the material (step S305).

[0042] The peak calculation process calculates the thickness of each material in the image based on the distance information extracted by the distance information extraction unit 260, and then sets the thickness of the search window 330 used to obtain the number of pixels of a straight line based on the calculated thickness.

[0043] (Correction method of the result correction unit 270) Figure 8 is an explanatory diagram showing an example of a method for correcting material boundary lines. Figure 9 is an explanatory diagram showing examples where material boundary lines are not detected or are falsely detected. The list of material boundaries calculated in step S206 may contain undetected or falsely detected material boundaries due to external factors. To correct these results, in step S209, a list of boundary intervals is created from the detected material boundary boundaries, and an approximation formula is created using this list to perform the following corrections.

[0044] (1) Areas with abnormally short intervals: These are judged to be false detections due to external factors such as shadows, and the corresponding boundary lines are deleted. (2) Areas with abnormally long intervals: It is determined that the boundary line could not be detected due to overexposure, etc., and the boundary line is added interpolatively.

[0045] This makes it possible to suppress the decrease in detection accuracy due to external factors such as lighting conditions and changes in the field of view during shooting (see Figure 8). In other words, the result correction unit 270 can create a more accurate material boundary list LA. The reason for creating the approximation formula in Figure 8 is that when shooting with a field of view, the spacing between material boundaries is not constant but decreases linearly. Therefore, the position of the appropriate boundary can be estimated from the approximation formula.

[0046] However, when creating this interval list, if there are many false positives or false negatives at the stage of creating the material boundary list, many incorrect interval values ​​(bold numbers) will be mixed into the interval list, as shown in Figure 9. If an approximation formula is created based on this list, the accuracy of the formula will decrease.

[0047] Therefore, in order to improve the accuracy of the approximation formula, the height of each material (reference value) registered in the material information 300 for counting is used in the material registration unit 220 for counting. In step S208 of Figure 5, the distance from the image acquisition device 100 to the material bundle 370 is calculated, and the ratio is calculated using the distance to the material bundle and the height of each material, which are pre-registered in the material information 300 for counting, to estimate the height (pixels) of each material shown in the image. By excluding values ​​outside a certain range from the interval list based on this estimated value, a highly accurate approximation formula is created, enabling more accurate counting.

[0048] In other words, the counting target material registration unit 220 of the counting device 200 stores distance information at the time of material registration (pre-registration of counting target materials) and height information per material at the time of material registration in its storage unit. The height information per material at the time of material counting is calculated based on the distance information at the time of material counting, the distance information at the time of material registration, and the height information per material at the time of material registration. The distance information extraction unit 260 extracts distance information between the target area and the mobile terminal based on the two-dimensional image information at the time of material counting.

[0049] Furthermore, the result correction unit 270 may perform a peak interval calculation process that calculates an appropriate peak interval according to the thickness of the material based on a list of peak values, a false detection determination process that estimates whether the material boundary is falsely detected or not by evaluating the amount of deviation between the actual peak interval and the appropriate peak interval, and a peak list correction process that corrects the list of peak values ​​based on the estimation result from the false detection determination process.

[0050] In addition to the above, the result of step S207 is used for correction in order to further improve accuracy. Figure 10 is an explanatory diagram showing an example of detecting a feature region on the front of a material. Specifically, after extracting the feature region on the front of the material as shown in Figure 10, the centroid coordinates of the extracted region are determined, and it is assumed that the material boundary line is located at the midpoint of the centroid coordinates (Y coordinates) of the upper and lower regions (the coordinate position of (Y1-Y2) / 2), and a material boundary line list (material boundary line list LB) is created. Then, compare material boundary list LA and material boundary list LB, and combine the results using AND or OR.

[0051] (Example of a mobile device screen) Next, examples of the screen display of the mobile terminal (image acquisition device 100) during material counting will be explained with reference to Figures 12A to 12C. Figure 12A shows an example screen when selecting counting materials on a mobile terminal and pressing the capture button. Figure 12B shows an example screen when uploading an image on a mobile terminal and displaying the counting results. Figure 12C shows an example screen when selecting an image counting result on a mobile terminal and displaying the detailed counting results.

[0052] The material bundle 181, described later, is a material bundle that replicates the actual site conditions, reproducing a situation where weight materials (without feature regions) are piled on top of the bundle. Regarding the counting process, only the materials with feature regions other than the weight materials are counted.

[0053] Screen 161 in Figure 12A is the display screen of the material selection unit 140 for counting. The display screen of the image acquisition device 100 shows the contents of the list of registered materials in the material information 300 for counting. In the example display, material 180A of thin material A and material 180B of thin river material B are displayed.

[0054] Screen 162 in Figure 12A is an example of the screen displayed after the user has selected (clicked) material 180A on screen 161 and has pressed the capture button. The image acquisition device 100 has a capture button 171 at the bottom of the screen.

[0055] Screen 163 in Figure 12B is an example of the screen displayed when uploading an image to the counting device 200. The bottom of the image acquisition device 100 has a "Count Now" button 173, a "Count Later" button 174, and a "Retake" button 175. The displayed screen shows the photographed bundle of materials 181. The "Count Now" button 173 is used to request counting from the counting device 200, while the "Count Later" button 174 is used to save the image to the photo folder of the image acquisition device 100 without requesting counting from the counting device 200.

[0056] Screen 164 in Figure 12B is an example of the screen displayed when the count result is shown after the "Count Now" button 173 is pressed on screen 163. The displayed screen shows the material bundle 181 that was just counted, as well as material bundles 182, 183, etc. that were counted in the past.

[0057] Screen 165 in Figure 12C is an example of the screen displayed when the user selects (clicks) the material bundle 181 on screen 164 and selects the count result. Screen 165 shows that 50 thin material A sheets have been counted. The image acquisition device 100 has an analysis details confirmation button 176 at the bottom of the screen.

[0058] Screen 166 in Figure 12C is an example of the screen displayed when the analysis details confirmation button 176 is pressed on screen 165, and the count results details are displayed. As seen on screen 166, the materials are displayed in different colors in groups of 10 (shown in shades of black and white in the figure), allowing the user to confirm the number of materials. As described above, the counting device 200 of this embodiment allows for easy counting of materials using a mobile terminal.

[0059] [Differentiation] The present invention is not limited to the embodiments described above, and various modifications are possible. The embodiments described above are illustrative examples provided to facilitate understanding of the present invention, and are not necessarily limited to those comprising all the configurations described. Furthermore, other configurations may be added to the configurations of the above embodiments, and some of the configurations may be replaced with other configurations. In addition, the control lines and information lines shown in the figures are those considered necessary for explanation, and do not necessarily represent all the control lines and information lines required in the product. In practice, it can be assumed that almost all configurations are interconnected. Possible modifications to the above embodiments are as follows, for example.

[0060] (1) In the above embodiment, a two-dimensional image is acquired from the image acquisition device 100, but it is not limited to this. In the case of a terminal in the image acquisition device 100 that has a three-dimensional sensor (not shown) such as LiDAR (Light Detection and Ranging), the image acquisition unit 110 may acquire two-dimensional image information and distance image information associated with the two-dimensional image information.

[0061] In other words, a counting device for counting the number of plates of material stacked in a predetermined direction, comprising: an image acquisition unit 110 that acquires two-dimensional image information and distance image information associated with the two-dimensional image information from a mobile terminal; a target material registration unit (count target material registration unit 220) that registers material areas in advance using two-dimensional image information; a target area recognition unit (count target area recognition unit 230) that recognizes the target area of ​​the material to be counted from the two-dimensional image information; a distance information extraction unit 260 that extracts distance information between the target area and the mobile terminal based on the distance image information; a boundary line detection unit 240 that detects (extracts) the boundary lines of the material within the target area; and a result correction unit 270 that determines whether the boundary lines are not detected or are falsely detected based on the extracted distance information and the interval of the detected boundary lines, and corrects the detection result of the boundary lines of the material.

[0062] (2) The processes shown in Figure 1 and the other processes described above were explained as software processes using a program in the above embodiment, but some or all of them may be replaced with hardware processes using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), etc. [Explanation of symbols]

[0063] 100 Image acquisition device 110 Image acquisition unit 120 Material position selection section 130 Data transmission unit 140 Countable Material Selection Section 150 Count Result Display Section 200 Counting device 210 Data receiving unit 220 Countable Material Registration Section (Target Material Registration Section) 221 Distance Estimation Processing Unit 222 Material Height Information Extraction Processing Unit 230 Count Target Area Recognition Unit (Target Area Recognition Unit) 240 Boundary line detection unit 250 Feature Region Detection Unit 260 Distance information extraction part 270 Result Correction Unit 280 Counting Result Registration Unit 300 items of material information that are counted. 310 Count Result Information 320 Materials Area 330 Search box 370 bundles of materials 980 Computer 981 CPU (Processing Unit) S120 Pre-registration process S220 Material Counting Processing

Claims

1. A counting device for counting the number of plates of material stacked in a predetermined direction, An image acquisition unit that acquires 2D image information from a mobile terminal, A target material registration unit that registers material areas in advance using the aforementioned two-dimensional image information, A target area recognition unit recognizes the target area of ​​the material to be counted from the two-dimensional image information, Based on the aforementioned two-dimensional image information, a distance information extraction unit extracts distance information between the target area and the mobile terminal using an AI learning model that describes the relationship between the distance between the target area and the mobile terminal and the monocular depth estimation of the two-dimensional image. A boundary detection unit for detecting the boundary line of the material within the target area, The system includes a result correction unit that determines whether a boundary line is not detected or is falsely detected based on the extracted distance information and the detected boundary line interval, and corrects the detection result of the boundary line of the material. A counting device characterized by the following features.

2. A counting device according to claim 1, further, It has a feature region detection unit that extracts feature regions on the surface of the material within the target region, The result correction unit determines whether a boundary line is not detected or is falsely detected based on the distance information, the extracted boundary line interval, and the feature area, and corrects the detection result of the boundary line of the material. A counting device characterized by the following features.

3. A counting device according to claim 1, The two-dimensional image information includes a material position selection unit in which the position of the material area is selected in advance. A counting device characterized by the following features.

4. A counting device according to claim 1, The aforementioned material registration unit stores distance information at the time of material registration and height information per material at the time of material registration in its storage unit. The height information per material when counting materials is calculated based on the distance information at the time of material counting, the distance information at the time of material registration, and the height information per material at the time of material registration. The distance information extraction unit extracts distance information between the target area and the mobile terminal based on the two-dimensional image information used during material counting. A counting device characterized by the following features.

5. A counting device according to claim 1, The boundary line detection unit performs a filtering process to remove unnecessary straight lines other than boundary lines included in the two-dimensional image information. For the image after the filtering process described above, the number of pixels in a horizontal line from the top edge of the image is obtained using a search window, a peak value is calculated based on the change in the number of pixels, and the coordinates corresponding to the peak value are extracted as material boundaries and listed in a peak calculation process. A counting device characterized by the following features.

6. A counting device according to claim 5, The peak calculation process calculates the thickness of each material in the image based on the distance information extracted by the distance information extraction unit, and sets the thickness of the search window used to acquire the number of pixels of a straight line based on the calculated thickness. A counting device characterized by the following features.

7. A counting device according to claim 5, The result correction unit performs a peak interval calculation process that calculates an appropriate peak interval according to the thickness of the material based on the list of peak values, A false detection determination process is performed to estimate whether a material boundary is falsely detected or not detected by evaluating the amount of deviation between the actual peak interval and the appropriate peak interval. Based on the estimation results from the false detection detection process, a peak list correction process is performed to modify the list of peak values. A counting device characterized by the following features.

8. A counting device for counting the number of plates of material stacked in a predetermined direction, An image acquisition unit that acquires two-dimensional image information and distance image information associated with said two-dimensional image information from a mobile terminal, A target material registration unit that registers material areas in advance using the aforementioned two-dimensional image information, A target area recognition unit recognizes the target area of ​​the material to be counted from the two-dimensional image information, A distance information extraction unit extracts distance information between the target area and the mobile terminal based on the distance image information, A boundary detection unit for detecting the boundary line of the material within the target area, The system includes a result correction unit that determines whether a boundary line is not detected or is falsely detected based on the extracted distance information and the detected boundary line interval, and corrects the detection result of the boundary line of the material. A counting device characterized by the following features.

9. A counting method for a counting device that counts the number of sheets of material stacked in a predetermined direction, The processing unit of the counting device is: Image acquisition step of obtaining 2D image information from a mobile terminal, A target material registration step in which the material area is registered in advance using the aforementioned two-dimensional image information, A target area recognition step in which the target area of ​​the material to be counted is recognized from the two-dimensional image information, Based on the aforementioned two-dimensional image information, a distance information extraction step is performed to extract distance information between the target area and the mobile terminal using an AI learning model that describes the relationship between the distance between the target area and the mobile terminal and the monocular depth estimation of the two-dimensional image. A boundary detection step for detecting the boundary line of the material within the target area, A result correction step is performed to determine whether a boundary line is not detected or is falsely detected based on the extracted distance information and the detected boundary line interval of the material, and to correct the detection result of the boundary line of the material. A counting method characterized by comprising a counting step of counting the number of materials based on the corrected detection result.

Images