Excavation site evaluation system, excavation site evaluation method, and excavation construction method

Abstract

To provide an excavation site evaluation system etc. capable of further efficiently discriminating stones of a predetermined size or larger to be removed at an excavation site.SOLUTION: An image of an excavation site 17 including a standard member 15 is imaged by an imaging apparatus 5 as image capturing means. Then, the excavation site image is corrected based on image information of the standard member 15 in the excavation site image by image correcting means. Then, the corrected image data is input to a learning model, above-mentioned feature quantities are extracted, and stones of a size equal to or larger than a predetermined size in the corrected image data are specified from these feature quantities. Then, the specified stone of the predetermined size or larger is displayed on a display unit 13, that is, the stone having the predetermined size or larger can be grasped.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an excavation site evaluation system and the like that enables efficient excavation work to be carried out at the site. [Background technology]

[0002] Conventionally, for example, when carrying out construction work for roads or buildings, or restoration work after a disaster, it has been necessary to excavate soil and sand at the site. At this time, excavation is carried out using heavy machinery, but stones and the like that are larger than a certain size must be sorted and crushed during excavation.

[0003] One such method of sorting is to use a skeleton bucket. A skeleton bucket has a lattice-shaped bottom. That is, holes of a predetermined size are formed in the bottom of the skeleton bucket. Therefore, any particles smaller than the holes in the soil scooped up by the skeleton bucket are sifted out through the holes, making it possible to sort out only stones and other particles larger than the predetermined size.

[0004] However, with this method, sieving is required even when there are no stones or other debris of a certain size or larger, and the sieved-out soil and sand must be excavated (collected) again using another heavy machine. Also, because the amount of soil and sand that can be sieved at one time depends on the size of the heavy machine, sieving the entire excavation site requires the use of large heavy machines or increased frequency of sieving, which results in poor work efficiency.

[0005] Another method is to use a large separate sieving screen instead of a skeleton bucket, but this requires that all of the excavated soil and gravel be dumped onto the screen, which creates the same problems as when using a skeleton bucket as described above.

[0006] On the other hand, instead of sifting through all the soil and sand, there is a method in which a person visually judges whether the stones are larger than a certain size and marks them, etc. The marked stones can then be crushed to a size smaller than the specified size using a breaker, etc., or only the targeted stones can be collected and piled up using other heavy machinery, etc.

[0007] However, since it is done visually by a person, a certain level of skill is required from the worker. Also, while the worker is on-site, heavy machinery cannot be used, so work must be suspended. Furthermore, if the excavation target is large, it requires many people and time.

[0008] In response to this, there is a method that uses image analysis. Image analysis makes it possible to calculate the size of stones from an image of the target and identify the size of each stone. In addition, a method that uses machine learning by artificial intelligence has also been proposed for this type of technique (for example, Patent Document 1). According to Patent Document 1, it is possible to determine the particle size distribution of excavated soil and sand, for example. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Patent Publication No. 2021-117625 Summary of the Invention [Problem to be solved by the invention]

[0010] However, determining particle size distributions using machine learning requires a large amount of learning data for various particle size distributions, and creating a learning model takes time.

[0011] Furthermore, when photographing from above the belt conveyor as in Patent Document 1, images can be acquired under more or less the same conditions, but images of an excavation site are greatly affected by the weather, time of day, the surrounding environment (presence or absence of mountain shadows, forests, etc.), and the unevenness of the ground. For this reason, even if the same learning model is used, for example, images taken on a cloudy evening and images taken on a sunny day at the same site may produce different results.

[0012] However, preparing learning data for each weather condition and time of day requires preparing many more learning images, which takes a lot of time to create a learning model.

[0013] The present invention has been made in consideration of such problems, and aims to provide an excavation site evaluation system, etc., that can more efficiently identify stones of a certain size or larger that require removal, etc. at excavation sites. [Means for solving the problem]

[0014] In order to achieve the above-mentioned object, a first invention is an evaluation system capable of evaluating the size of stones at an excavation site, comprising: a learning model that uses an image of the excavation site for learning as input information and learns information about stones of a predetermined size or larger that are present at the excavation site as output information; a standard member for image correction that is placed at a predetermined position at the excavation site; an image acquisition means that acquires an excavation site image including the standard member; an image correction means that corrects the excavation site image based on image information about the standard member in the excavation site image; a stone identification means that inputs corrected image data corrected by the image correction means to the learning model and identifies stones of a predetermined size or larger in the corrected image data; and a display means that displays the identified stones of a predetermined size or larger. The image acquisition means is disposed on a heavy machine that excavates the excavation site, the standard member is disposed near the tip of the arm of the heavy machine, and the standard member can be moved to a predetermined position on the ground within the field of view of the image acquisition means by the heavy machine, and the image acquisition means can acquire the excavation site image in a state where the standard member has been moved to the predetermined position within the field of view of the image acquisition means, and the image correction means can determine unevenness or inclination of the ground at the excavation site from the shape of the standard member in the excavation site image taken in a state where the standard member has been moved to one or more locations, and correct the excavation site image to a flat image. The present invention is a drilling site evaluation system characterized by the above.

[0015] The image correction means may have a light meter placed at the excavation site, and may be capable of correcting the excavation site image based on image information of the standard component in the excavation site image and illuminance information obtained by the light meter.

[0019] Further, a first invention may be an evaluation system capable of evaluating the size of stones at an excavation site, comprising: a learning model that uses images of the excavation site for learning as input information and learns information about stones present at the excavation site that are a predetermined size or larger as output information; an image acquisition means for acquiring images of the excavation site within a predetermined range; a machine guidance function that can grasp the position of the tip of the arm of a heavy machine when the tip of the arm is moved to a predetermined position on the ground at the excavation site; an image correction means that determines the unevenness or slope of the ground at the excavation site based on the position information of the tip of the arm and corrects the excavation site image to a flat image; a stone identification means that inputs the corrected image data corrected by the image correction means into the learning model and identifies stones in the corrected image data that are a predetermined size or larger; and a display means that displays the identified stones that are a predetermined size or larger.

[0020] The excavation site may include an illuminance meter arranged at the excavation site, and the image correction means may be further capable of correcting the excavation site image based on illuminance information obtained by the illuminance meter.

[0021] The image correction means may be capable of further performing orthorectification on the excavation site images.

[0022] The stone identification means may be capable of identifying the type of stone and of identifying stones that are equal to or larger than a size that is pre-associated with each type of stone.

[0023] According to the first invention, by using a learning model based on images of an excavation site, information on stones of a predetermined size or larger can be obtained efficiently. In this case, instead of outputting the particle size distribution of the soil and sand, only stones of a predetermined size or larger are extracted from the image. Therefore, learning data for a wide range of particle size distribution from small to large diameters is not required, and the learning data can be simplified.

[0024] In addition, by capturing images of standard components along with the excavation site, and then performing various corrections based on the standard components in the excavation site image and standardizing the excavation site image, it is possible to reduce the effects of weather and time at the time of image acquisition, as well as the effects of unevenness and slopes at the excavation site, thereby making it possible to estimate stone information more accurately.

[0025] For example, if the standard component is a colored staff, the standard color (e.g., RGB, black and white, etc.) of the colored staff component shown in the excavation site image can be extracted, and the color balance can be corrected so that each color of the colored staff in the image matches the standard color. Therefore, even if the color balance of the image changes depending on the weather, time, etc., this effect can be minimized and stone information can be extracted with high accuracy.

[0026] Furthermore, the shape of the standard component in the image may be used to determine the unevenness or slope of the ground. For example, when a standard component is placed in a predetermined location within the captured image range, if the shape of the standard component in the image deviates from the shape of the standard component that would appear if the excavation site were completely flat, it can be determined that the excavation site is uneven or sloped. Therefore, the excavation site image can be corrected to an image of a flat surface. For example, a stone on an uphill slope toward the back may appear large in the excavation site image even if it is actually small. However, by suppressing this effect, stone information can be more accurately estimated.

[0027] Furthermore, by attaching such a standard member to the vicinity of the tip of the arm of a heavy machine (for example, a bucket) and moving the standard member to various locations, there is no need for a person to manually place the standard member at various locations in the excavation site (image capture range) and then remove it. This makes it possible to efficiently grasp the unevenness and slopes at various locations within a predetermined range (image capture range) of the excavation site.

[0028] Alternatively, by using heavy machinery with machine guidance functions, the tip of the arm can be placed directly on the ground to determine the elevation of the ground in question. Therefore, by standardizing the excavation site images, the influence of unevenness and slopes at the excavation site can be reduced, allowing for more accurate estimation of stone information.

[0029] In addition, by using a light meter to correct the image according to the illuminance, the brightness and contrast of the excavation site image can be appropriately corrected, and evaluation can be performed based on the standardized corrected image.

[0030] Furthermore, by performing ortho-correction on the excavation site images, it is possible to obtain a more visual understanding of the placement of stones, etc.

[0031] Furthermore, if the learning model can identify the type of stone from its shape, etc., it can link the size of the stone to be extracted to each type of stone. For example, stones that can be easily crushed with a heavy machinery bucket can be handled with the same heavy machinery used for excavation, even if they are relatively large.

[0032] The second invention is an excavation site evaluation method capable of evaluating the size of stones at an excavation site, which uses an image of the excavation site for learning as input information, creates a learning model that learns information about stones of a predetermined size or larger that are present at the excavation site as output information, places a standard member for image correction at a predetermined position at the excavation site, and displays an image of the excavation site including the standard member. By imaging device Get The imaging device is placed on a heavy machine that excavates an excavation site, the standard member is placed near the tip of the arm of the heavy machine, and the heavy machine can move the standard member to a predetermined position on the ground within the field of view of the imaging device, and images of the excavation site are acquired while the standard member is moved to a predetermined position within the field of view of the imaging device, and images are taken while the standard member is moved to one or more locations. Based on image information of the standard component in the excavation site image, The unevenness or inclination of the ground at the excavation site is determined from the shape of the standard member, The image of the excavation site On a flat image This is an excavation site evaluation method, which is characterized by correcting the image data, inputting the corrected image data into the learning model, identifying stones in the corrected image data that are larger than a predetermined size, and displaying the identified stones that are larger than the predetermined size.

[0033] According to the second invention, stones of a predetermined size or larger at an excavation site can be identified with high accuracy through simple learning.

[0034] The third invention is an excavation construction method using the excavation site evaluation method of the second invention, characterized in that it repeats the steps of removing or crushing stones identified as being of a predetermined size or larger, excavating the excavation site, and identifying stones of a predetermined size or larger at the excavation site after excavation using the excavation site evaluation method.

[0035] According to the third invention, it is possible to evaluate the stones at the excavation site in real time while excavation work is being carried out at the excavation site. [Effects of the Invention]

[0036] According to the present invention, it is possible to provide an excavation site evaluation system etc. that can more efficiently identify stones of a predetermined size or larger that require removal etc. at an excavation site. [Brief explanation of the drawings]

[0037] [Figure 1] 1 is a side view showing an excavation site evaluation system 10. FIG. [Figure 2] 1 is a plan view showing an excavation site evaluation system 10. FIG. [Figure 3] 1 is a diagram showing the configuration of an excavation site evaluation system 10. FIG. [Figure 4] 3 is a flowchart of an excavation site evaluation method using the excavation site evaluation system 10. [Figure 5] (a) is a conceptual diagram of image 23, and (b) is a conceptual diagram of image 23a. [Figure 6] FIG. 1 is a side view showing an excavation site evaluation system 10a. [Figure 7] 10 is a flowchart of an excavation site evaluation method using the excavation site evaluation system 10a. [Figure 8] FIG. 10 is a side view showing the excavation site evaluation system 10b. [Figure 9] 1 is a flowchart of another excavation site evaluation method. DETAILED DESCRIPTION OF THE INVENTION

[0038] An excavation site evaluation system according to a first embodiment will be described below. Figures 1 and 2 are schematic diagrams showing the use of an excavation site evaluation system 10 capable of evaluating the size of stones at an excavation site, with Figure 1 being a side view and Figure 2 being a plan view. Figure 3 is a diagram showing the configuration of the excavation site evaluation system 10. In the present invention, "stone" refers to anything that can be separated from the ground (rock mass) without being integrated with it, regardless of its size, and includes what is called rock or gravel.

[0039] The excavation site evaluation system 10 mainly comprises a GPS 3, an imaging device 5, an illuminance meter 7, a display unit 13, and a processing unit 11 that performs various processes to control these. The GPS 3 that can acquire position information of the excavation unit (heavy equipment 1), the imaging device 5 that is an image acquisition means, the illuminance meter 7 that can measure the illuminance of the excavation site 17, the processing unit 11, and the display unit 13 are installed on the heavy equipment 1. Note that some or all of the GPS 3, the imaging device 5, the illuminance meter 7, the display unit 13, and the processing unit 11 may be installed in a location other than the heavy equipment 1 and connected to each other by wire or wirelessly.

[0040] The heavy equipment 1 has a bucket at the tip of the arm 9 that can excavate the ground. The imaging range 21 of the imaging device 5 is the range in which the bucket can excavate the ground by, for example, extending and retracting the arm 9. A standard member 15 for image correction is placed in the imaging range 21. The standard member 15 may be placed anywhere within the imaging range 21, but for example, the standard member 15 in the imaging range 21 is placed so that it is in a predetermined position (for example, the upper right of the captured image). Details of the standard member 15 will be described later. The position information acquired by the GPS 3, the excavation site image captured by the imaging device 5, and the illuminance information from the illuminometer 7 are sent to the processing unit 11.

[0041] As shown in FIG. 3, the processing unit 11 is, for example, a computer, and is composed of a control unit 33, a memory unit 35, a media input / output unit 37, a communication control unit 39, an input unit 41, a peripheral device I / F unit 45, etc., which are connected via a bus 47.

[0042] The control unit 33 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The CPU loads programs stored in the storage unit 35, the ROM, a recording medium, etc. into a work memory area on the RAM and executes them, drives and controls each device connected via the bus 47, and realizes various processes performed by the excavation site evaluation system 10 as described below.

[0043] The storage unit 35 is an HDD (hard disk drive) or SSD (flash SSD) (solid state drive), and stores programs executed by the control unit 33, data necessary for program execution, an OS (operating system), etc. These program codes are read out as needed by the control unit 33 and transferred to RAM, and then read out by the CPU and executed as various means. The storage unit 35 also stores a learning model and various data, which will be described later.

[0044] The media input / output unit 37 (drive device) inputs and outputs data, and includes media input / output devices such as a floppy (registered trademark) disk drive, a CD drive, a DVD drive, an MO drive, a USB memory, and an SD card.

[0045] The communication control unit 39 has a communication control device, a communication port, etc., and is a communication interface that mediates communication between the computer and the network.

[0046] The input unit 41 inputs data and includes input devices such as a keyboard, a pointing device such as a mouse, a numeric keypad, etc. Operation instructions, action instructions, data input, etc. can be given to the computer via the input unit 41.

[0047] The display unit 13 has a display device such as a liquid crystal panel, a logic circuit (video adapter, etc.) for implementing the video function of a computer in cooperation with the display device, etc. The display unit 13 is installed, for example, in the cockpit of the heavy equipment 1 so that the operator can grasp the display in real time. The display unit 13 may also be a hologram lens or a projection device onto the windshield of the cockpit.

[0048] The peripheral device I / F (interface) unit 45 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 45.

[0049] The bus 47 is a path for transmitting and receiving control signals, data signals, etc. between the devices. The excavation site evaluation system 10 is not limited to one including all of the above components, but may have only the components necessary to perform the functions of the present invention.

[0050] Next, the learning model stored in the storage unit 35 will be described. The learning model used in the present invention uses a plurality of learning excavation site images as input information, and has been machine-learned to learn information about stones of a predetermined size or larger that exist at the linked excavation sites. More specifically, the sizes of stones of a certain size are measured at various excavation sites, and the relationship between the feature amounts of each excavation site image and the information about stones of a predetermined size or larger is learned as a discrimination model.

[0051] As mentioned above, conventional methods for measuring the particle size distribution of soil and sand using image processing involve quantifying the particle size of each particle or grouping particles into groups within a specified particle size range and then performing statistical processing to determine the distribution. However, this method requires measurement and learning over a wide range of particle sizes, from fine to large.

[0052] In contrast, in this embodiment, it is sufficient to be able to identify stones of at least a predetermined size. In other words, rather than learning the degree of "grain size" as in learning grain size distribution, learning is performed to output information on stones of a predetermined size or larger that exceed a certain threshold, as in the case of making a pass / fail judgment on the presence or absence of defects. This allows for a reduction in the amount of data used for learning.

[0053] When learning information about stones of a predetermined size or larger from an image of an excavation site, a known feature extraction means is used to extract a predetermined number of feature amounts from the image of the excavation site, and stones are identified based on these feature amounts, and information about stones of a predetermined size or larger is linked to them. The feature amounts in the image include the color, density (brightness), distribution, texture, etc. of each pixel in the image. From these image feature amount elements, stones are recognized based on gloss and shadow (contrast). Furthermore, the shape and size of the stone, changes in the image feature amount elements as a whole (uniformity or non-uniformity), and the shape and distribution of the changed parts of the image feature amount elements can also be used as feature amounts.

[0054] It should be noted that when acquiring images of the drilling site, special infrared cameras or hyperspectral cameras are expensive and require special analysis, making them undesirable. It is also desirable to be able to evaluate characteristics from images taken with ordinary digital cameras, without using stereo or 3D images.

[0055] Next, we will explain the excavation site evaluation method by the excavation site evaluation system 10. Figure 4 is a flowchart of the excavation site evaluation method for evaluating the size of stones at an excavation site by the excavation site evaluation system 10. As described above, first, a learning model is created by linking a plurality of learning excavation site images with information on stones of a predetermined size or larger (step 101).

[0056] In this case, the excavation site image for learning may be a corrected image obtained by performing the correction described below on an image of the excavation site. That is, a predetermined correction may be performed on an image of the actual excavation site, and this corrected image may be used as learning data. In addition, stones of a predetermined size or larger in the learning data may be identified and linked by conventional sieving or the like at the excavation site.

[0057] With such a learning model stored in the memory unit 35, illuminance information of the excavation site 17 is acquired by the illuminance meter 7 placed at the excavation site 17 (heavy equipment 1) (step 102), and an image of the excavation site 17 including the standard component 15 is captured by the imaging device 5, which is an image acquisition means (step 103).

[0058] Next, based on the illuminance information obtained by the illuminometer 7, the image correction means (control unit 33) corrects the excavation site image using a preset method (step 104). For example, if the illuminance information indicates an illuminance above a predetermined level, the shadows of stones and the like will be intensified and the excavation site image will be bright overall. Therefore, the image correction means (control unit 33) performs image correction to reduce the brightness of the excavation site image and weaken the contrast of the shadows. On the other hand, if the illuminance information indicates an illuminance below a predetermined level, the entire image will be dark and the boundaries between shadows and the like will likely become unclear. Therefore, the image correction means (control unit 33) performs image correction to increase the brightness of the excavation site image and strengthen the contrast of the shadows. Note that if correction by the illuminance meter 7 is not required, steps 102 and 104 can be omitted.

[0059] Next, in a similar manner, the image correction means (control unit 33) corrects the excavation site image based on the image information of the standard member 15 in the excavation site image. In this embodiment, the standard member 15 is a color staff member having a reference color staff. The color staff member is colored with the reference color staff. For example, the color staff member is a standard color such as each of the RGB colors and black. The standard color of the color staff member is not particularly limited, and may be one or more colors, and it is desirable to have at least three RGB colors. In addition, the surface of the color staff member is subjected to an anti-reflection treatment to suppress color bleeding due to reflection of sunlight, etc.

[0060] The image correction means (control unit 33) acquires color information for each standard color of the color staff members appearing in the obtained excavation site image (corrected image based on illuminance information), and performs image correction based on the acquired color information so that these colors become standard color tones (step 105). Note that the standard color tones may be the color tones of each standard color in the image when the color staff members are imaged in advance under specific conditions, or the actual color of the color staff members themselves may be the standard color tones. In this way, the control unit 33 (image correction means) corrects the excavation site image so that, for example, the RGB color balance matches the color balance in the standard color tones.

[0061] As mentioned above, the excavation site images used for learning when creating a learning model may also be corrected in the same way. This allows the learning data and the image data to be evaluated to be corrected under the same conditions, thereby reducing the effects of variations in images caused by factors such as imaging conditions.

[0062] In this way, the image correction means (control unit 33) can correct the excavation site image based on the image information of the standard member 15 in the excavation site image and the illuminance information obtained by the illuminometer 7. The captured excavation site image and the corrected image are stored in the memory unit 35. At this time, each excavation site image may be linked to location information obtained by the GPS 3.

[0063] Next, the corrected image data is input into the learning model, the aforementioned feature quantities are extracted, and stones in the corrected image data that are larger than a predetermined size are identified from these feature quantities (step 106). For example, after recognizing the short side of the extracted stone shape, calculations are performed to determine how many pixels the short side is and how many mm one pixel is based on a standard component, and then the stone size is recognized, and as a result, stones that are larger than a predetermined size can be identified.

[0064] Next, the control unit 33 causes the display unit 13 to display the identified stones of a predetermined size or larger (step 107). Fig. 5(a) is a conceptual diagram showing an image 23 displayed on the display unit 13. As shown in Fig. 5(a), on the display unit 13, only specific stones 19a are displayed by coloring or the like in the image (corrected image) captured by the imaging device 5 at the excavation site 17 that includes stones 19. In other words, stones 19a of a predetermined size or larger can be identified.

[0065] As mentioned above, the display unit 13 may not be a display but may be projected onto the windshield or the like, or the stones 19a may be identified by irradiating the stones at the actual site with a laser or the like.

[0066] The image correction means (control unit 33) may further perform orthorectification on the excavation site image. Fig. 5(b) is a conceptual diagram showing an image 23a after orthorectification. In this way, the control unit 33 may display only stones 19a of a predetermined size or larger in an identifiable manner in the orthoimage.

[0067] As a method for excavating the excavation site 17, first, the stones 19a identified in this manner that are larger than a predetermined size are removed or crushed. For example, the identified stones 19a are crushed or removed from the excavation site 17 to a collection point using other heavy machinery. After the identified stones 19a are crushed or removed from the excavation site 17, an image of the excavation site may be acquired again to check whether any stones 19a larger than the predetermined size remain or whether any new stones 19a have been exposed.

[0068] Thereafter, excavation of the excavation site 17 is carried out. After the excavation of the predetermined range is completed, the excavation site 17 after excavation is checked for the presence of stones 19a of a predetermined size or larger using the excavation site evaluation method described above. If stones of a predetermined size or larger are present, they are crushed or removed, and excavation work is carried out. The above steps are repeated to excavate the predetermined range of the excavation site 17.

[0069] As described above, according to this embodiment, stones of a predetermined size or larger can be identified in real time at the excavation site, so it is easy to determine which stones need to be crushed, etc., and excavation work becomes easier. In addition, by placing standard members 15 at the excavation site 17, identifying the standard members 15 in the captured image, and correcting the color balance of the captured image using the color tone of the standard members 15, it is possible to reduce the influence of the imaging conditions on the captured image. Therefore, stones of a predetermined size or larger can be identified with high accuracy.

[0070] Furthermore, by adding correction based on illuminance information from an illuminometer, it is possible to correct the captured image with higher accuracy.

[0071] For example, by creating a learning model that takes into account the variations in the captured images, it is possible to estimate the color tone and brightness of the excavation site from various feature quantities in the captured images, but this requires learning images with different brightness and color tones.In contrast, in this embodiment, by using standardized images, the learning model can be made simpler and the accuracy of estimation can be improved.

[0072] Furthermore, the displayed image may be a specific stone 19a displayed in an image in the field of view seen by the operator of the heavy machinery, or may be displayed in a planar view by ortho-correction.

[0073] Next, a second embodiment will be described. Fig. 6 is a diagram showing an excavation site evaluation system 10a according to the second embodiment. In the following description, explanations that overlap with those of the first embodiment will be omitted.

[0074] In this embodiment, the standard member 15a is not a colored staff member, but a member of a predetermined shape (for example, the standard member itself may be spherical or of another shape, or may be provided with a specific pattern or the like), and is placed at the excavation site 17. The standard member 15a is placed at one or more predetermined positions within the field of view (imaging range) of the imaging device 5, but it is preferable to place the standard member 15a at multiple positions. For example, the standard members 15a are placed at predetermined intervals, such as at the four corners and the center of the imaging range.

[0075] As described above, the imaging device 5 is mounted on the heavy equipment 1. When imaging the excavation site 17 with the imaging device 5 installed on the heavy equipment 1 in this way, the image is taken from an oblique direction, so that stones 19 at the back appear relatively small and stones 19 closer appear relatively large. Even in this case, when imaging is always performed at a flat imaging site, the size of the stones can be determined by their position in the captured image (for example, the top is the back and the bottom is the front) taking into account the distance from the imaging device 5.

[0076] However, if there are unevenness or slopes at the excavation site 17, it is difficult to determine the size of the stone in the captured image. For example, in Fig. 6, if the virtual ground when the excavation site 17 is flat is designated as B, then in the viewing direction A (i.e., a predetermined position in the captured image), position D of the standard member 15a is captured in the same direction as position C on the virtual ground B. That is, in the captured image, C and D are captured at the same position (the same vertical position in the image).

[0077] In this way, if there are unevenness or slopes at the excavation site 17, the size of the stone 19 at position C on the virtual ground B and the size of the stone 19 at position D on the actual ground will actually be different, even if they appear to be the same in the captured image. For example, the stone 19 at the back of the captured image (e.g., the top of the captured image) is located far from the imaging device 5, so this is taken into account when learning and determining the size of the stone 19. However, if the size of the stone 19 is determined as being at position C, the stone 19 will appear larger at position D because it is closer to the imaging device 5, and there is a risk that the stone 19 will be erroneously determined to be larger than it actually is.

[0078] In contrast to this, in this embodiment, the image correction means (control unit 33) determines the form specified by the standard member 15a in the captured image (the size and shape of the standard member itself, and the size and shape of the pattern or the like attached to the standard member) and calculates the distance from the imaging device 5, thereby making it possible to determine whether the position of the standard member 15a is position C or position D. By calculating this for multiple points in the imaging range, it becomes possible to roughly determine the unevenness or slope of the ground at the excavation site 17.

[0079] Next, the site evaluation method in this embodiment will be described in detail. Fig. 7 is a flowchart of the excavation site evaluation method by the excavation site evaluation system 10a. First, as in the first embodiment, a learning model is created using an image of the excavation site 17 as input information and information on stones of a predetermined size or larger as output information (step 201). At this time, a corrected image, which will be described later, can also be used as the image for learning.

[0080] Next, an image of the target excavation site including the standard member is acquired (step 202), and as described above, the unevenness or inclination of the ground at the excavation site 17 is determined based on the shape of the standard member 15a in the excavation site image, and the excavation site image is corrected to a flat image (step 203).

[0081] For example, the image correction means (control unit 33) calculates the unevenness or slope of the ground at the excavation site 17 from the size or shape (i.e., how it appears) of the standard member 15a in the excavation site image. Thereafter, for example, in FIG. 6, the image of the stone 19 at position D in the captured image is corrected to a position on the virtual ground B where the image capture device 5 is rotated around the center while the distance between the image capture device 5 and D is kept constant. In other words, the stone 19 at position D in the captured image is moved to the front side (to the bottom of the image).

[0082] This corrected image is input to the learning model, and stone information specifying stones of a predetermined size or larger is output (step 204), and displayed on the display unit 13 (step 205). The display method on the display unit 13 is the same as in the first embodiment.

[0083] In this way, by using an image in which the unevenness and slopes of the excavation site 17 have been corrected to appear flat, the size of the stone can be identified with greater accuracy.

[0084] The standard member 15a may be installed in a bucket or the like at the tip of the arm 9 instead of being installed directly at the excavation site 17. That is, by placing the standard member 15a near the tip (bucket or the like) of the arm 9 of the heavy equipment 1, the heavy equipment 1 can move the standard member 15a to a predetermined position on the ground within the field of view of the imaging device 5. Fig. 8 is a diagram showing an excavation site evaluation system 10b in which the standard member 15a is installed near the tip of the arm 9.

[0085] As shown in the figure, the standard member 15a is placed at a predetermined position on the ground by bringing the bucket into contact with the ground. In this case, too, the difference between position C on the virtual ground B in the direction of field of view A from the imaging device 5 (i.e., a predetermined position in the captured image) can be grasped by the size or shape of the standard member 15a.

[0086] In this case, the imaging device 5 acquires each excavation site image while moving the standard member 15a to a predetermined position in the field of view range (for example, the four corners and the center of the imaging range). After this, the image correction means (control unit 33) combines the respective excavation site images captured while the standard member 15a is moved to multiple locations. In other words, a composite image is created in which the standard member 15a is captured in multiple locations on a single excavation site image.

[0087] Then, the image correction means (control unit 33) determines the unevenness or inclination of the ground at the excavation site from the size or shape of the standard member 15a in the excavation site image from the obtained composite image, and corrects the excavation site image to a flat image. In this way, a corrected image can be obtained. Note that if the standard member 15a is placed in one location, image synthesis is not necessary.

[0088] Note that a portion of the bucket shape may be used as the standard member 15a. Instead of combining multiple images captured by moving the standard member 15a to each location, height information at each position of the standard member 15a may be acquired from each image, and the acquired height information may be applied to an excavation site image that does not include the standard member 15a, thereby performing similar correction. Furthermore, if a machine guidance function capable of grasping the position of the tip of the arm of a heavy machine when moving the tip of the arm (bucket) to a predetermined position on the ground at the excavation site is provided, the bucket's position information (e.g., elevation) can be grasped. Therefore, the relative positional relationship (separation and angle) between the imaging device 5 and the bucket can be used to grasp the unevenness and inclination of the ground at the excavation site 17. Based on the positional information of the tip of the arm, the unevenness or inclination of the ground at the excavation site can be determined, and the excavation site image can be corrected to a flat image.

[0089] As described above, according to the second embodiment, the same effects as those of the first embodiment can be obtained. Furthermore, by determining the unevenness and slope of the ground and performing plane correction, it is possible to estimate the stone size with higher accuracy. In this way, by using standardized and corrected images as the images input to the learning model, it is possible to suppress variations in the imaging conditions and excavation site conditions in non-standardized images taken as they were, thereby enabling highly accurate estimation. Furthermore, by performing machine learning using corrected images that have been standardized and corrected for learning images, it is possible to create a learning model with higher accuracy.

[0090] The first and second embodiments may be combined, for example, color correction or illuminance (brightness) correction may be combined with plane correction.

[0091] Furthermore, if learning images containing multiple types of stones are used, it is possible to estimate the type of stone based on the shape, color, and presence or absence of shading of the stones in the image. In this case, different criteria may be used to link information on stones of a predetermined size or larger to each type of stone.

[0092] 9 is a flowchart showing an excavation site evaluation method for estimating stone size according to stone type. First, a learning model is created using an image of the excavation site as input information, and stone type information and information on stones larger than a predetermined size as output information (step 301). That is, the control unit 33 (stone identification means) uses stone information including stone type as input information and output stone information, not just those larger than a predetermined size.

[0093] Next, using the same procedure as in the first embodiment, an image of the excavation site including the standard member 15 (15a) is acquired (step 302), and the excavation site image is corrected using the standard member 15 (15a) in the excavation site image (step 303).

[0094] Next, the corrected image is input into a learning model, and stone information identifying stone sizes above a predetermined size determined by the type of stone is output (step 304). By linking the stone type with the identified stone size in advance, the stone type in the captured image can be identified and the size can be identified for each stone type. For example, the extraction size may be larger for rock masses that can be easily crushed or removed with a bucket than the extraction size for relatively round, normal stones. Since such rock masses are easily crushed and reduced in size during excavation work without the need for crushing with other heavy machinery, the threshold size may be larger. In this way, if the type of stone can be identified, stones above the size previously associated with each stone type can be identified from the excavation site image.

[0095] Thereafter, for each type of stone, stones of a predetermined size or larger are displayed on the display unit 13. In other words, only stones that need to be crushed or removed before excavation can be identified and displayed on the display unit 13. In this way, by linking the size of stones to be extracted with each type of stone, stones of a size or larger according to the type of stone can be identified.

[0096] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the technical scope of the present invention is not limited to the above-described embodiments. It is clear that those skilled in the art can conceive of various modifications and alterations within the scope of the technical ideas described in the claims, and it is understood that these modifications and alterations also fall within the technical scope of the present invention.

[0097] For example, the imaging device 5 is not limited to being installed on heavy machinery, but may be installed on other supports or on flying objects such as drones. Furthermore, if it is not necessary to obtain location information, a GPS is not necessarily required.

[0098] 1. Heavy machinery 3...GPS 5. Imaging device 7……Luminance meter 9...Arm 10, 10a, 10b... Excavation site evaluation system 11...Processing section 13...Display section 15, 15a...Standard parts 17... Excavation site 19, 19a……stone 21....Image range 23, 23a...Image

Claims

1. An evaluation system capable of evaluating stone size at an excavation site, A learning model that uses an image of an excavation site for learning as input information and learns information about stones of a predetermined size or larger that are present at the excavation site as output information; A standard member for image correction to be placed at a predetermined position at the excavation site; an image acquisition means for acquiring an image of an excavation site including the standard member; an image correction means for correcting the excavation site image based on image information of the standard member in the excavation site image; a stone identification means for inputting the corrected image data corrected by the image correction means into the learning model and identifying stones in the corrected image data that are equal to or larger than a predetermined size; a display means for displaying the identified stones of a predetermined size or larger; Equipped with the image acquisition means is disposed in a heavy machine that excavates an excavation site, the standard member is disposed near the tip of an arm of a heavy machine, and the standard member can be moved to a predetermined position on the ground within the field of view of the image acquisition means by the heavy machine; the image acquisition means is capable of acquiring the excavation site image in a state in which the standard member is moved to a predetermined position within a field of view of the image acquisition means, The image correction means is capable of determining the unevenness or inclination of the ground at the excavation site from the shape of the standard member in the excavation site images taken while the standard member is moved to one or more locations, and correcting the excavation site images to flat images.

2. An evaluation system capable of evaluating stone size at an excavation site, A learning model that uses an image of an excavation site for learning as input information and learns information about stones of a predetermined size or larger that are present at the excavation site as output information; image acquisition means for acquiring an image of a predetermined area of ​​an excavation site; a machine guidance function that can grasp the position of the tip of the arm of the heavy equipment when the tip of the arm is moved to a predetermined position on the ground at the excavation site; an image correction means for determining unevenness or inclination of the ground at the excavation site based on position information of the tip of the arm and correcting the excavation site image to a flat image; a stone identification means for inputting the corrected image data corrected by the image correction means into the learning model and identifying stones in the corrected image data that are equal to or larger than a predetermined size; a display means for displaying the identified stones of a predetermined size or larger; An excavation site evaluation system comprising:

3. having a light meter located at the excavation site; 3. The excavation site evaluation system according to claim 1, wherein the image correction means is further capable of correcting the excavation site image based on illuminance information obtained by the illuminometer.

4. 3. The excavation site evaluation system according to claim 1, wherein the image correction means is capable of further performing ortho-correction on the excavation site images.

5. 3. The excavation site evaluation system according to claim 1, wherein the stone identification means is capable of identifying the type of stone and of identifying stones that are larger than a size that is pre-associated with each type of stone.

6. An excavation site evaluation method capable of evaluating the size of stones at an excavation site, comprising: A learning model is created that uses images of an excavation site for learning as input information and information about stones of a predetermined size or larger that are present at the excavation site as output information. Standard components for image correction are placed at designated locations on the excavation site. Acquire an excavation site image including the standard member using an imaging device; the imaging device is disposed on a heavy machine excavating an excavation site, the standard member is disposed near the tip of an arm of a heavy machine, and the standard member can be moved to a predetermined position on the ground within the field of view of the imaging device by the heavy machine; The standard member is moved to a predetermined position within the field of view of the imaging device, and the excavation site image is acquired. Based on image information of the standard member in the excavation site images captured while the standard member is moved to one or more locations, the unevenness or inclination of the ground at the excavation site is determined from the shape of the standard member, and the excavation site image is corrected to a flat image; The corrected image data is input into the learning model, and stones having a predetermined size or larger are identified in the corrected image data. A method for evaluating an excavation site, characterized in that stones of a specified size or larger are displayed.

7. An excavation construction method using the excavation site evaluation method according to claim 6, removing or crushing stones identified as being equal to or larger than a predetermined size; conducting excavation at the excavation site; a step of identifying stones of a predetermined size or larger at the excavation site after excavation by the excavation site evaluation method; An excavation method characterized by repeating the above steps.

Images