SOIL PROPERTY ESTIMATION METHOD, TRAINED MODEL GENERATION METHOD, SOIL PROPERTY ESTIMATION DEVICE, AND PROGRAM

A machine learning-based method using photographic and detection data enhances the accuracy and efficiency of soil property estimation during construction projects.

JP7580275B2Active Publication Date: 2024-11-11PENTA OCEAN CONSTRUCTION CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2021003642
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-13
Publication Date
2024-11-11
Estimated Expiration
2041-01-13

AI Technical Summary

Technical Problem

Conventional methods for determining soil quality and rock type during construction projects lack accuracy and are time-consuming, relying on preliminary boring surveys and surface appearances.

Method used

A method involving machine learning-based soil property estimation using photographic data and detection data such as sound and vibration, processed through a trained model to enhance accuracy.

Benefits of technology

Enables more accurate and timely estimation of soil characteristics by integrating image and sound/vibration data, reducing the effort and time required for soil property determination.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007580275000001
    Figure 0007580275000001
  • Figure 0007580275000002
    Figure 0007580275000002
  • Figure 0007580275000003
    Figure 0007580275000003
Patent Text Reader

Abstract

To estimate properties of soil materials including soils, rock kinds, etc. in a highly accurate manner.SOLUTION: A soil property estimation method using artificial intelligence is provided, including: a step (S101) of obtaining photographing data, using the photographing data representing images of an estimation subject ground and detection data representing at least either one of sound or vibration occurred by work against the estimation subject ground; a step (S102) of obtaining the detection data; and a step (S103) of estimating soil properties of the ground based on output data obtainable by inputting the photographing data and detection data into a trained model built by machine learning.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for estimating soil properties. [Background technology]

[0002] In civil engineering works such as excavation or embankment works of ground or bedrock, the soil type and rock type at the work site are judged. Conventionally, the soil type and rock type are judged based on the results of a prior boring survey, the appearance of the ground surface, and strength and hardness information obtained by a portable cone penetration tester or a soil hardness tester.

[0003] Patent document 1 also describes a system that acquires two-dimensional image data including information about areas of materials on the earth's surface, and uses the acquired two-dimensional image data to classify the material of the ground within a specified region and period. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-220058 A Summary of the Invention [Problem to be solved by the invention]

[0005] Although conventional methods can classify the ground material, there is room for improvement in terms of estimating the characteristics of soil materials, such as soil type and rock type, with greater accuracy.

[0006] The present invention has been made in view of the above problems, and has an object to estimate soil properties of ground with higher accuracy. [Means for solving the problem]

[0007] In order to solve the above problems, a soil characteristic estimation method according to one embodiment of the present invention includes a step of acquiring photography data representing an image of the ground that is the target of construction work, a step of acquiring detection data representing at least one of sound and vibration generated by the construction work, and a step of estimating the soil characteristics of the ground using output data obtained by inputting the photography data and the detection data into a trained model constructed by machine learning, performed by an information processing device.

[0008] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting the photographed data and the detected data to the trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0009] In the above soil characteristic estimating method, the detection data may be data indicating a result of spectrum analysis of time-series data indicating at least one of sound and vibration generated by the construction work.

[0010] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting data indicating the results of spectrum analysis of time-series data representing sound and / or vibration to the trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0011] In the above soil property estimating method, the detection data may be image data representing a result of spectrum analysis of the time-series data.

[0012] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting image data representing the results of spectrum analysis of time-series data representing sound and / or vibration to a trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0013] In the above-mentioned soil characteristic estimation method, the step of acquiring the detection data may include a step of removing a predetermined frequency band of sound or vibration recorded by a recording device during the construction work, and the data from which the predetermined frequency band has been removed may be acquired as the detection data.

[0014] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting detection data from which frequency bands of unnecessary sounds or vibrations generated during construction work have been removed into a trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0015] In the above soil property estimating method, the removing step may remove a frequency band corresponding to an operation sound or an operation vibration of a work machine performing the construction work.

[0016] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting detection data from which frequency bands corresponding to the operating sounds or vibrations of a work machine performing construction work have been removed into a trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0017] The above soil characteristic estimation method may further include a step of acquiring operation data representing information regarding the operation of a work machine performing the construction work, and in the estimating step, the photographing data, the detection data and the operation data may be input into the trained model.

[0018] According to the above configuration, the information processing device estimates the soil properties of the ground using output data obtained by inputting operation data representing information related to the operation of a work machine performing construction work into a trained model, thereby making it possible to estimate the soil properties with higher accuracy.

[0019] The above soil characteristic estimation method may further include a step of acquiring first data including photography data representing an image of the ground that is the target of construction work and detection data representing at least one of sound and vibration generated by the construction work, and second data indicating characteristics of the soil material of the ground, and a step of generating the trained model by machine learning the correlation between the first data and the second data using the first data and the second data corresponding to the first data.

[0020] According to the above configuration, the information processing device generates a trained model by machine learning using the first data including the photographing data and the detection data and the second data indicating the characteristics of the soil material. By using the trained model, the information processing device can estimate the soil characteristics with higher accuracy.

[0021] In the above soil characteristic estimation method, in the step of acquiring the first data and the second data, shooting time information indicating the shooting timing of the shooting data and shooting position information indicating the shooting position corresponding to the shooting data are acquired, and detection time information indicating the detection timing of the detection data and detection position information indicating the detection position corresponding to the detection data are acquired, and the shooting data and the detection data may be matched based on the shooting time information and the detection time information, and the shooting position information and the detection position information.

[0022] According to the above configuration, the information processing device generates a trained model using the photographed data and the detected data associated with each other based on the time information and the location information. By using the trained model, the information processing device can estimate soil properties with higher accuracy.

[0023] In addition, the trained model generation method of the present invention includes a step of acquiring, by an information processing device, first data including photography data representing an image of the ground that is the target of construction work and detection data representing at least one of sound and vibration generated by the construction work, and second data indicating soil characteristics of the ground, and a step of generating a trained model by machine learning the correlation between the first data and the second data using the first data and the second data corresponding to the first data.

[0024] According to the above configuration, the information processing device generates a trained model by machine learning using the first data including the photographing data and the detection data and the second data indicating the properties of the soil material. By using the trained model, it is possible to estimate the soil properties with higher accuracy.

[0025] In addition, the soil characteristic estimation device of the present invention includes a first acquisition unit that acquires photography data representing an image of the ground that is the target of construction work, a second acquisition unit that acquires detection data representing at least one of sound and vibration generated by the construction work, and an estimation unit that estimates the soil characteristics of the ground using output data obtained by inputting the photography data and the detection data into a trained model constructed by machine learning.

[0026] According to the above configuration, the soil characteristics estimation device estimates the soil characteristics of the ground using output data obtained by inputting the photographed data and the detected data into the trained model, thereby making it possible to estimate the soil characteristics with higher accuracy.

[0027] Furthermore, the program according to the present invention is a program for causing a computer to function as the soil property estimation device, and is a program for causing a computer to function as the first acquisition unit, the second acquisition unit, and the estimation unit.

[0028] According to the above configuration, the soil characteristics estimation device estimates the soil characteristics of the ground using output data obtained by inputting the photographed data and the detected data into the trained model, thereby making it possible to estimate the soil characteristics with higher accuracy. Effect of the Invention

[0029] According to one aspect of the present invention, the soil properties of the ground can be estimated with greater accuracy. [Brief description of the drawings]

[0030] [Figure 1] 1 is a diagram showing an overview of a soil property estimation system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the configuration of the main parts of various devices included in the soil characteristic system according to the first embodiment of the present invention. [Diagram 3] FIG. 1 is a diagram illustrating an example of a trained model according to a first embodiment of the present invention. [Figure 4] 3 is a sequence chart showing the flow of a soil property estimating method according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing an example of imaging data according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing an example of a waveform represented by detection data according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram illustrating an example of the contents of detection data converted by spectral analysis. [Figure 8] FIG. 11 is a block diagram showing the configuration of the main parts of various devices included in the soil characterization system according to the second embodiment of the present invention. [Figure 9] 11 is a sequence chart showing the flow of a trained model generation method according to a second embodiment of the present invention. [Figure 10] 11 is a sequence chart showing the flow of a soil property estimating method according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] [Embodiment 1] <System Overview> Fig. 1 is a diagram showing an overview of a soil properties estimation system 1 according to this embodiment. The soil properties estimation system 1 is a system that estimates the soil properties of the ground that is the target of construction work. Construction work is work that brings about some kind of change to the ground or bedrock, such as excavation work, cutting work, filling work, leveling work, and compaction work. The ground is the land that serves as the foundation that supports buildings and structures, and includes the bedrock on the ground surface.

[0032] The ground may be a layer of soil material of a single soil or rock type, or may be a layer of multiple layers. For example, the ground may be a layer of gravel laid on top of soil. For example, the ground to be worked on includes a new surface layer of 30 cm thickness that is being worked on by leveling or compacting.

[0033] The soil characteristics of the ground refer to the characteristics and / or properties of the ground. Examples of the soil characteristics of the ground include the soil quality, rock type, strength, hardness, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and / or the amount of moisture (water content) of the soil material contained in the ground. Examples of the soil material include rock, soil, sand, clay, or rock materials such as gravel and pebbles. The soil characteristics can be determined by an expert with reference to various measurement data. In this embodiment, the "expert" refers to a person who has the skills to determine the soil characteristics, such as a geological expert or a geological engineer. In addition, the soil characteristics are measured, for example, by a portable cone penetration tester or a soil hardness tester.

[0034] The soil property estimation system 1 includes an information processing device 10, a measuring device 20, and a work machine 30. The information processing device 10 and the measuring device 20 are connected to each other via a communication path N1 so as to be able to communicate with each other.

[0035] The work machine 30 is a machine that performs construction work, and examples thereof include a backhoe, a bulldozer, a payloader, or a roller. Examples of construction work performed by the work machine 30 include excavation work using a backhoe, and leveling and compaction work using a payloader, a bulldozer, or a roller.

[0036] The measuring device 20 is a device for measuring the ground, which is the target of construction work. In this embodiment, the measuring device 20 takes an image of the ground, which is the target of construction work, as a subject. The measuring device 20 also generates data representing the sound and / or vibration generated by the construction work.

[0037] The information processing device 10 (an example of a soil property estimation device) is a device that estimates the soil material of the ground using a trained model. Here, the trained model is a model that has been machine-learned to learn the correlation between the ground photography data and detection data and the soil property judgment results.

[0038] The communication path N1 is a transmission path for wireless communication and / or wired communication, such as Bluetooth (registered trademark), a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these. However, the configuration of the communication path N1 is not limited to these.

[0039] ≪Main part configuration≫ (Measuring Equipment) 2 is a block diagram showing the configuration of the main parts of the devices included in the soil property estimation system 1. The measuring device 20 includes a control unit 21, a communication unit 22, an imaging unit 23, a detection unit 24, a light source 25, and a storage unit .

[0040] (Photography Department) The photographing unit 23 photographs the ground. For example, the photographing unit 23 may be a camera for photographing an image for generating a stereo image or a three-dimensional image of the ground. A "stereo image" is an image that can be viewed stereoscopically, generated from a plurality of monocular images from different viewpoints, or a group of a plurality of monocular images. A "three-dimensional image" is a three-dimensional stereoscopic image generated from a plurality of monocular images. For example, the photographing unit 23 may be a camera for photographing an image for generating a spectral image of the ground. A "spectral image" is an image that records electromagnetic waves of a plurality of wavelength bands, generated from at least one of a plurality of images that record electromagnetic waves of different wavelength bands acquired by a spectral camera or the like. In a spectral image, electromagnetic waves of at least one of wavelength bands such as visible light, ultraviolet light, infrared light, and far infrared light are recorded.

[0041] The photographing unit 23 is arranged and fixed so as to be able to photograph the ground, which is the target of the work, during construction work. As an example, the photographing unit 23 is preferably installed in a position that is less affected by vibration and where it is easy to photograph the working state of the work machine 30 (for example, the working state of the claws of a backhoe, etc.). For example, when the work machine 30 is a machine that works without moving, such as a backhoe, the photographing unit 23 is fixed to the work machine 30. On the other hand, when the work machine 30 is a machine that works while moving or vibrating, such as a bulldozer or roller, the photographing unit 23 may be fixed to the work machine 30 using a gimbal and a shock-absorbing buffer member as necessary.

[0042] The photographing unit 23 performs so-called "time lapse photography" in which continuous photographs are taken at regular time intervals during construction work such as continuous excavation work.

[0043] (Detection unit) The detection unit 24 detects at least one of sound and vibration generated by construction work. Sound generated by construction work includes, for example, sound generated from the ground due to construction work, or sound generated from the work machine 30. Vibration generated by construction work includes, for example, vibration from the work machine or vibration of the ground. As an example, the detection unit 24 is a sound collection device that collects sound generated by construction work.

[0044] One example of the sound generated from the ground due to construction work is the sound generated when the soil is compacted during soil spreading work. Such sounds are related to the way the soil is compacted and the type of soil, and are influenced by, for example, the fine grain content, water content, grain size, mineral composition, etc. As another example, compaction work is a vertical movement work, and since sounds are generated from the soil and rocks contained in the ground that is the target of the work, the detection unit 24 detects and collects these sounds.

[0045] The frequency characteristics of the sounds and vibrations that occur depend on the type of rock or soil. For example, hard rocks tend to be dominated by high frequencies, while soft soil tends to be dominated by low frequencies. Also, differences in rocks and soils result in differences in the characteristics of the frequency bands that appear as a first dominant frequency band, a second dominant frequency band, and so on.

[0046] The detection unit 24 is arranged and fixed so as to be able to detect at least one of sound and vibration generated by construction work. As one example, the detection unit 24 is fixed to the work machine 30. As one example, if the detection unit 24 is a sound collection device, the detection unit 24 is fixed to the work machine 30 so that the sound collection direction of the detection unit 24 is the direction of the ground photographed by the photographing unit 23. In addition, the operating directions of the photographing unit 23 and the detection unit 24 may be linked.

[0047] (light source) The light source 25 is a standard light source when the image capturing unit 23 captures an image. The light source 25 can be realized by, for example, a light using a light-emitting diode. It is desirable that the light of the light source 25 has a color and intensity similar to that of sunlight. For example, it is desirable that the light of the light source 25 has a saturation within a predetermined range. Here, the predetermined range indicates, for example, a range of saturation values ​​that appear as white light when the light of the light source 25 is viewed by the human eye. Also, for example, the light of the light source 25 may be light of a hue that appears as white light. White light is whitish light that has almost no hue, and may be, for example, light of a color with a saturation below a threshold or a color temperature of 3500 Kelvin or more and 5000 Kelvin or less. This makes it possible to minimize the influence of the color of the light of the light source 25 on the captured image. It is also desirable that the light source 25 is a flat light source that can irradiate diffuse light from a flat surface. This makes it possible to irradiate light evenly during shooting.

[0048] (Control unit) The control unit 21 performs overall control of the measuring device 20. The control unit 21 controls the photographing unit 23 to photograph the ground. For example, the control unit 21 may cause the photographing unit 23 to perform photographing in response to a photographing start instruction received from another device via the communication unit 22. Also, for example, the control unit 21 may accept a user's input operation to instruct the start of photographing via an input device or the like provided in the measuring device 20 or connected to the measuring device 20. Then, when the input operation is accepted, the control unit 21 may cause the photographing unit 23 to perform photographing.

[0049] The control unit 21 generates imaging data representing an image of the ground that is the target of the construction work, based on the image captured by the imaging unit 23. The imaging data may be data representing an image such as a stereo image or a three-dimensional image, or may be data representing a spectral image. The control unit 21 also generates detection data representing the sound and / or vibration detected by the detection unit 24.

[0050] As an example, the photographing by the photographing unit 23 and the sound collection by the detection unit 24 are performed at the same timing, more preferably at approximately the same time. Also, a necessary portion may be cut out from the continuously collected detection data and used. The control unit 21 may associate the photographing data and the detection data obtained in one measurement. The control unit 21 may store the photographing data and the detection data in the storage unit 26 for each measurement.

[0051] Furthermore, the control unit 21 may read out the photographed data and detection data stored in the storage unit 26 in response to a request from the information processing device 10, and transmit the photographed data and detection data to the information processing device 10 via the communication unit 22. Furthermore, the control unit 21 may autonomously transmit the photographed data and detection data to the information processing device 10 for each measurement.

[0052] Furthermore, in the case where the measurement device 20 includes a light source 25, the control unit 21 may control at least one of the position and the orientation of the light source 25. In the case where the measurement device 20 includes a light source 25, the control unit 21 may control the intensity of the light from the light source 25.

[0053] Furthermore, the control unit 21 generates time information of the video, sound and / or vibration based on a predetermined time signal. Furthermore, the control unit 21 generates position information indicating the position of the video, sound and / or vibration based on the predetermined signal. The position information includes coordinate values ​​(x, y, z), for example. The control unit 21 generates the coordinate values ​​(x, y, z), for example, based on a signal received by a satellite positioning system (Global Navigation Satellite System, etc.). Note that the method of generating the position information is not limited to the above-mentioned method, and other methods of generating position information may be used.

[0054] For example, the control unit 21 may generate position information by a method using an automatic tracking total station. In this case, as an example, a marker is installed on the work machine 30, and the marker is installed at a predetermined reference position. A predetermined photographing device (not shown) takes images so that both the reference point and the work machine 30 are included in the photographing range, and a device such as the measuring device 20 may perform image analysis of the photographed images to generate position information.

[0055] The control unit 21 assigns shooting time information and shooting position information to the shooting data. The shooting time information is information indicating the timing of shooting the shooting data. As an example, the shooting time information is information indicating the timing when the control unit 21 generated the shooting data. The shooting position information is information indicating the shooting position. As an example, the shooting position information includes coordinate values ​​(x, y, z) generated by the control unit 21.

[0056] The control unit 21 assigns detection time information and detection position information to the detection data. The detection time information assigned to the detection data is information indicating the detection timing of the sound and / or vibration, and is, for example, information indicating the timing at which the control unit 21 generated the detection data. The detection position information is information indicating the detection position of the sound and / or vibration represented by the detection data. For example, the detection position information includes coordinate values ​​(x, y, z) generated by the control unit 21.

[0057] (Storage part) The storage unit 26 is a storage device that stores various data necessary for processing by the measurement device 20. The storage unit 26 temporarily stores the photographed data and the detected data generated by the control unit 21.

[0058] (Communications Department) The communication unit 22 is a communication interface that communicates between the measuring device 20 and other devices. The communication unit 22 communicates with other devices such as the information processing device 10 under the control of the control unit 21. As an example, the communication unit 22 transmits the imaging data and detection data generated by the control unit 21 to the information processing device 10. The communication unit 22 may also receive an instruction to start imaging in the measuring device 20 from another device (not shown). When the instruction to start imaging is received, the communication unit 22 outputs the instruction to the control unit 21.

[0059] 2 illustrates the measuring device 20 including one image capturing unit 23 and one detector 24, but the measuring device 20 may include multiple image capturing units 23 and / or multiple detectors 24. As an example, the measuring device 20 may be configured to include multiple image capturing units, such as a first image capturing unit that is a stereo camera and a second image capturing unit that is an infrared camera. As another example, the measuring device 20 may be configured to include multiple detectors, such as a first detector that collects sound and a second detector that detects vibrations. The measuring device 20 may also include an input unit such as a button, a mouse, and a touch panel, and a display unit such as a display.

[0060] (Information processing device) (Communications Department) The information processing device 10 includes a control unit 11, a communication unit 12, and a storage unit 13. The communication unit 12 is a communication interface that communicates between the information processing device 10 and other devices. The communication unit 12 communicates with other devices such as a measuring device 20 under the control of the control unit 11. As an example, the communication unit 12 receives photographed data and detected data from the measuring device 20. Note that when the photographed data and detected data are acquired via an external recording medium, the communication unit 12 does not need to communicate with the measuring device 20.

[0061] (Storage part) The memory unit 13 is a storage device that stores data necessary for processing by the information processing device 10. The memory unit 13 stores a program for executing a method for estimating soil characteristics by the information processing device 10. The memory unit 13 also stores a trained model LM1. The trained model LM1 is a trained model that has been machine-learned to obtain a correlation between the photographing data and the detection data and the judgment result of the soil characteristics. The structure of the trained model LM1 is not particularly limited. For example, the trained model LM1 can be realized by a model having a CNN (Convolutional Neural Network) structure.

[0062] The storage unit 13 may be an external device of the information processing device 10. For example, the storage unit 13 may be a storage device such as a server communicably connected to the information processing device 10. The information processing device 10 may also have an interface to which an external recording medium can be connected. For example, the information processing device 10 may have an interface to which a USB flash memory or an SD card can be connected. The information processing device 10 may also include an input unit such as a button, a mouse, and a touch panel, and a display unit such as a display.

[0063] (Control unit) The control unit 11 comprehensively controls the information processing device 10. The control unit 11 includes a first acquisition unit 111, a second acquisition unit 112, and an estimation unit 113.

[0064] (1st acquisition part) The first acquisition unit 111 acquires the imaging data generated by the measuring device 20. As an example, the first acquisition unit 111 may acquire the imaging data by receiving the imaging data from the measuring device 20. In addition, as an example, the first acquisition unit 111 may acquire the imaging data by reading the imaging data from a storage device such as an external storage device in which the imaging data is stored.

[0065] (Second acquisition part) The second acquisition unit 112 acquires the detection data generated by the measuring device 20. As an example, the second acquisition unit 112 may acquire the detection data by receiving the detection data from the measuring device 20. In addition, as an example, the second acquisition unit 112 may acquire the detection data by reading the detection data from a storage device such as an external storage device in which the detection data is stored.

[0066] (Estimation Department) The estimation unit 113 estimates the soil properties of the ground using output data obtained by inputting the photographing data acquired by the first acquisition unit 111 and the detection data acquired by the second acquisition unit 112 into the learned model LM1.

[0067] <Details of trained model> 3 is a diagram illustrating an example of the trained model LM1 according to the present embodiment. Input data including image data and detection data is input to the trained model LM1.

[0068] The trained model LM1 is composed of, for example, a convolutional layer, a pooling layer, and a combination layer. In the convolutional layer, the input data is convoluted by filtering. The convolutional data is pooled in the pooling layer. This improves the model's ability to recognize changes in the position of features in the data. The pooled data is processed in the combination layer, and is converted into the output data of the trained model LM1, i.e., the format of the estimated results of soil properties, and output.

[0069] In other words, by passing the photographed data and detected data input as input data to the trained model LM1 through each layer shown in Figure 3 in the order shown, the estimated results of the soil properties are finally output as output data.

[0070] The output data of the trained model LM1 is data indicating soil characteristics. For example, the output data includes information indicating rock type or soil type, such as "sandy soil", "clay soil", "soft rock I", and "soft rock II". For example, the output data includes information indicating the strength and hardness of the ground. The output data may also include a cone penetration resistance value that is an index of the degree of compaction of the ground.

[0071] The output format of the estimation results is not particularly limited. For example, various characteristics may be shown as numerical index values, or may be shown as binary values ​​such as "moisture present" or "absence of moisture." Alternatively, the estimation results of soil properties may be shown as text data.

[0072] <Processing flow> 4 is a sequence chart showing the flow of the soil property estimation method S100 according to this embodiment. As an example, the soil property estimation method S100 is executed while the work machine 30 is performing construction work on the ground. Note that some steps may be executed in parallel or in a different order. Prior to executing the soil property estimation method S100, it may be confirmed whether the timekeeping by the measurement device 20 is set correctly.

[0073] (S101) In step S101, the measuring device 20 captures an image of the ground that is the target of the construction work, and generates image data. The image capturing unit 23 performs time lapse photography, thereby generating a plurality of pieces of image data. The image capturing data is, for example, data representing a stereo image, a three-dimensional image, or a spectral image based on an image captured by the image capturing unit 23 fixed to the work machine 30.

[0074] Fig. 5 is a diagram showing an example of an image represented by the photographing data. In the figure, the ground in image Img11 and the ground in image Img12 are mainly composed of clay. The ground in image Img13 is mainly composed of rocks. The clay that makes up image Img11 is softer than the clay that makes up the ground in image Img12. As shown by these examples, the characteristics of the soil materials that make up the ground vary depending on the type, etc.

[0075] (S102) 4, the measuring device 20 detects sound and / or vibration generated by the construction work with the detection unit 24 and generates detection data representing the detected sound and / or vibration. The detection data is data representing sound and / or vibration detected during the construction work at a timing corresponding to the timing of photographing the ground or continuously including the timing of photographing.

[0076] (S103) In step S103, the measuring device 20 transmits the generated shooting data to the information processing device 10. The first acquisition unit 111 of the information processing device 10 receives the shooting data from the measuring device 20 via the communication unit 12, and stores the received shooting data in the storage unit 13.

[0077] (S104) In step S104, the measurement device 20 transmits the generated detection data to the information processing device 10. The second acquisition unit 112 of the information processing device 10 receives the detection data from the measurement device 20 via the communication unit 12, and stores the received detection data in the memory unit 13.

[0078] 6 is a diagram showing an example of a waveform represented by detection data. In the diagram, a waveform s11 is a waveform of a sound picked up by the detection unit 24. In the diagram, the horizontal axis represents time (seconds) and the vertical axis represents sound pressure (dB).

[0079] (S105) In step S105 of FIG. 4, the second acquisition unit 112 of the information processing device 10 performs a spectrum analysis such as a fast Fourier transform (FFT) on the detection data acquired in step S104, and converts the detection data into spectrum data for each frequency. In this embodiment, the converted spectrum data is used as the detection data to be input to the trained model LM1. In other words, the detection data to be input to the trained model LM1 is data indicating the result of a spectrum analysis of time-series data representing at least one of sound and vibration generated by construction work. The second acquisition unit 112 stores the converted detection data in the memory unit 13.

[0080] 7(a) and (b) are diagrams illustrating the contents of detection data converted by spectral analysis. In FIG. 7(a), the horizontal axis indicates time (seconds) and the vertical axis indicates frequency (Hz). Also, in FIG. 7(a), the magnitude of amplitude is indicated by color. In FIG. 7(b), the horizontal axis indicates frequency and the vertical axis indicates amplitude. FIG. 7(b) is two-dimensional information indicating the relationship between frequency and amplitude. FIG. 7(a) shows the results of spectral analysis performed at every predetermined elapsed time, and is three-dimensional information indicating the relationship between elapsed time, frequency, and amplitude.

[0081] (S106·S107) 4, the first acquisition unit 111 acquires the shooting data by reading the shooting data from the storage unit 13. Also, in step S107, the second acquisition unit 112 acquires the detection data by reading the detection data from the storage unit 13. The shooting data acquired by the first acquisition unit 111 and the detection data acquired by the second acquisition unit 112 are input data to be input to the trained model LM1.

[0082] (S108) In step S108, the first acquisition unit 111 (or the second acquisition unit 112) associates the captured image data and the detection data in which the image time information corresponds to the detection time information and the image position information corresponds to the detection position information. As an example, the first acquisition unit 111 (or the second acquisition unit 112) associates the captured image data and the detection data in which the difference between the assigned time information is less than a predetermined threshold and the difference between the assigned position information is less than a predetermined threshold from among the multiple captured image data and the multiple detection data stored in the storage unit 13.

[0083] The first acquisition unit 111 (or the second acquisition unit 112) may associate the shooting data with the detection data on a one-to-one basis, or may associate them on a multiple-to-multiple basis. The first acquisition unit 111 (or the second acquisition unit 112) may also associate multiple pieces of shooting data with one piece of detection data. As an example, the first acquisition unit 111 (or the second acquisition unit 112) may associate a series of multiple pieces of shooting data captured by time-lapse photography with data indicating the results of a spectrum analysis of sounds continuously collected during the shooting period of the shooting data. In other words, the input data of the trained model LM1 includes one or more pieces of shooting data and one or more pieces of detection data.

[0084] (S109) In step S109, the estimation unit 113 of the information processing device 10 estimates the soil properties of the ground using output data obtained by inputting the photographed data and the detected data into the trained model LM1. At this time, the estimation unit 113 may use the output data of the trained model LM1 as the estimation result as it is, or may generate data indicating the estimation result using the output data of the trained model LM1. As an example, the estimation unit 113 may generate data representing a three-dimensional configuration diagram of soil and rock layers at a construction work site based on the output data of the trained model LM1 and position information corresponding to the photographed data and the detected data.

[0085] (S110) In step S110, the estimation unit 113 of the information processing device 10 outputs the estimation result of step S109. As an example, the estimation unit 113 may display the estimated soil characteristics on a display unit such as a display. As another example, the estimation unit 113 may output data indicating the estimated soil characteristics by transmitting the data to another terminal at a construction work site or the like via the communication path N1. As another example, the estimation unit 113 may output data indicating the estimated soil characteristics by storing the data in an external storage device.

[0086] As described above, according to this embodiment, in estimating soil properties using the machine-learned trained model LM1, data representing not only images but also sounds and / or vibrations are used as input data. This allows soil properties to be estimated more accurately than when only images are used as input data.

[0087] Conventionally, the method of determining soil properties in construction work has been based on the results of a prior boring survey, the appearance of the ground surface, and strength and hardness information obtained by a portable cone penetration tester or a soil hardness tester. Such conventional methods have the problem that determining soil properties requires time and effort. In contrast, according to the present embodiment, it is possible to reduce the time and effort required to estimate soil properties.

[0088] In this embodiment, the information processing device 10 collects photographed data and detected data collected at the construction work site in real time, and estimates soil properties using output data obtained by inputting the collected data into the trained model LM1. This makes it possible to estimate the soil properties of the ground in real time in parallel with the progress of construction work by the work machine 30.

[0089] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0090] (Main components of the device) 8 is a block diagram showing the configuration of the main parts of the devices included in the soil property estimation system 1B according to the present embodiment. The soil property estimation system 1B according to the second embodiment includes an information processing device 10B instead of the information processing device 10 according to the first embodiment. The information processing device 10B includes a control device 11B instead of the control device 11 of the information processing device 10. The control device 11B includes an estimation phase execution unit 110 and a learning phase execution unit 120. The estimation phase execution unit 110 includes a first acquisition unit 111, a second acquisition unit 112, and an estimation unit 113. The learning phase execution unit 120 includes a teacher data acquisition unit 121 and a generation unit 122.

[0091] The teacher data acquisition unit 121 acquires teacher data, which is a set of first data corresponding to the input of the trained model LM1 and second data corresponding to the output of the trained model LM1. The first data includes photographed data and detected data. The contents of the photographed data and detected data are similar to those shown in the above-mentioned embodiment 1. The photographed data included in the first data is, for example, image data showing the ground photographed by the photographing unit 23 fixed to the work machine 30. The detected data included in the first data is, for example, data showing the results of a spectrum analysis of the sound and / or vibration detected by the detection unit 24. The photographed data and detected data include time information and position information, as shown in the above-mentioned embodiment 1.

[0092] For example, the teacher data acquiring unit 121 may acquire the first data by receiving the first data from the measurement device 20. Also, for example, the teacher data acquiring unit 121 may acquire the first data by reading the first data from an external storage device in which the first data is stored.

[0093] The second data is data showing the soil characteristics of the ground. The second data is, for example, data showing the results of a judgment made by an expert with reference to various measurement data related to the ground. The second data is, for example, data showing the results of a judgment made based on the results of a prior boring survey, the appearance of the ground surface, and strength and hardness information obtained by a portable cone penetration tester or a soil hardness tester.

[0094] The second data includes location information indicating a location of the ground that is the subject of the soil property assessment, and may also include time information indicating a time when construction work was carried out on the ground that is the subject of the assessment.

[0095] As an example, the teacher data acquisition unit 121 may acquire the second data input by an expert using an input unit (mouse, touch panel, etc.) of the information processing device 10B, or may acquire the second data by receiving the second data input by an expert operating another terminal from the other terminal via the communication path N1 by the information processing device 10B. Also, the teacher data acquisition unit 121 may acquire the second data by reading the second data from a storage device such as an external storage device in which the second data is stored.

[0096] The teacher data acquisition unit 121 associates the first data with the second data corresponding to the first data and stores them in the memory unit 13. In the memory unit 13, a set of the first data and the second data is stored as one teacher data. That is, the teacher data is composed of a combination of photographed data and detected data related to the ground that is the target of the construction work, and the judgment results of the soil characteristics of the ground.

[0097] The generation unit 122 uses the first data and second data corresponding to the first data to generate a trained model LM1 that has undergone machine learning of the correlation between the first data and the second data.

[0098] (Flow of trained model generation method) 9 is a sequence chart illustrating the flow of a trained model generating method S200 performed by the information processing device 10 B. Note that some steps may be executed in parallel or in a different order.

[0099] (S201) In step S201, while the work machine 30 is performing construction work on the ground, the measurement device 20 photographs the ground that is the target of the construction work, and generates photographed data.

[0100] (S202) In step S202, the measuring device 20 detects sound and / or vibration generated by the construction work with the detection unit 24 and generates detection data representing the detected sound and / or vibration. The detection data is data representing sound and / or vibration detected at a timing corresponding to the timing of photographing the ground, or continuously including the timing of photographing.

[0101] (S203) In step S203, the measurement device 20 transmits the imaging data and the detection data to the information processing device 10B. The teacher data acquisition unit 121 of the information processing device 10B receives the imaging data and the detection data, and stores the received imaging data and the detection data in the storage unit 13. In other words, the teacher data acquisition unit 121 acquires first data including the imaging data and the detection data.

[0102] The teacher data acquisition unit 121 may receive the imaging data and the detection data separately, or may receive the imaging data and the detection data together. The teacher data acquisition unit 121 may also receive a plurality of imaging data and / or a plurality of detection data together. The teacher data acquisition unit 121 executes one or more reception processes of the imaging data and / or the detection data. As a result, a plurality of imaging data and a plurality of detection data are accumulated in the memory unit 13.

[0103] (S204) In step S204, the teacher data acquisition unit 121 performs a spectrum analysis on the received detection data, and converts the detection data into detection data that represents a spectrum for each frequency. The teacher data acquisition unit 121 stores the converted detection data in the storage unit 13.

[0104] (S205) In step S205, the teacher data acquisition unit 121 acquires the second data. As an example, the teacher data acquisition unit 121 acquires the second data by receiving the second data from another terminal via the communication path N1.

[0105] (S206) In step S206, the teacher data acquisition unit 121 associates the shooting data with the detection data based on the time information and the position information. As an example, the teacher data acquisition unit 121 associates the shooting data with the detection data based on the shooting time information and the shooting position information given to the shooting data and the detection time information and the detection position information given to the detection data, and sets the associated shooting data and detection data as first data.

[0106] In addition, the teacher data acquisition unit 121 associates the first data with the second data based on the location information assigned to each data. At this time, the teacher data acquisition unit 121 may associate the first data with the second data based on the location information and time information.

[0107] (S207) In step S207, the generation unit 122 uses the first data and second data corresponding to the first data to generate a trained model LM1 that has undergone machine learning of the correlation between the first data and the second data.

[0108] (Flow of soil property estimation method) The flow of the soil property estimating method according to this embodiment is similar to the flow of the soil property estimating method S100 described in the first embodiment above, and therefore the description thereof will not be repeated here.

[0109] According to the present embodiment, the information processing device 10B constructs a trained model LM1 using a set of the first data and the second data. The trained model LM1 constructed by the information processing device 10B is used to estimate the soil properties of the ground that is the target of the construction work, so that the soil properties can be estimated with higher accuracy.

[0110] Furthermore, according to the present embodiment, the information processing device 10B associates the captured image data with the detected data based on time information and position information, thereby reducing the effort required to create teacher data.

[0111] [Embodiment 3] A third embodiment of the present invention will be described with reference to the drawings. Note that components having the same functions as those described in the first and second embodiments above are given the same reference numerals, and the description thereof will not be repeated.

[0112] A soil property estimation system 1C according to this embodiment includes an information processing device 10C instead of the information processing device 10B according to the second embodiment. In this embodiment, the detection unit 24 of the measurement device 20 is a recording device that records sounds or vibrations generated by construction work. Note that the configuration of the information processing device 10C is not shown in the figure because it has the same configuration as the information processing device 10 shown in FIG. 2.

[0113] 10 is a sequence chart showing the flow of the soil property estimating method S100C according to this embodiment. The soil property estimating method S100C includes a process of step S111 after step S104 of the soil property estimating method S100 shown in FIG.

[0114] In step S111, the second acquisition unit 112 removes a predetermined frequency band of sounds or vibrations picked up by the detection unit 24 during construction work. The predetermined frequency band is a frequency band corresponding to sounds or vibrations that are considered unnecessary in the analysis of soil characteristics. As an example, the predetermined frequency band is a frequency band corresponding to the operating sounds or operating vibrations of the work machine 30 performing construction work.

[0115] Specifically, for example, the detection unit 24 collects in advance the mechanical sounds of the work machine 30 when no construction work is being performed, and a device such as the information processing device 10C performs spectrum analysis such as FFT on the collected sound data to analyze the frequency band. The second acquisition unit 112 processes the sound data by filtering out the frequency band of the mechanical sounds of the work machine 30 for the sounds collected by the detection unit 24 during construction work.

[0116] After completing the process of step S111, in step S105, the second acquisition unit 112 generates data indicating a spectrum analysis result of the detection data from which the predetermined frequency band has been removed, and stores the data in the storage unit 13. After that, in step S107, the second acquisition unit 112 acquires the detection data by reading it out from the storage unit 13. In other words, in step S107, the second acquisition unit 112 acquires the detection data from which the predetermined frequency band has been removed.

[0117] According to this embodiment, the information processing device 10C estimates the soil properties of the ground using output data obtained by inputting detection data from which frequency bands of unnecessary sounds and vibrations (noise) generated during construction work have been removed into the trained model LM1. When the sounds and vibrations generated by the work machine 30 are large as noise, the estimation accuracy can be improved by removing the sounds and vibrations. In other words, according to this embodiment, the soil properties can be estimated with higher accuracy than when the noise is not removed.

[0118] [Additional Notes] [Additional Note 1] The data included in the input data to be input to the trained model LM1 is not limited to those shown in the above-described embodiment. In other words, the input data may include other data in addition to the photographed data and the detected data.

[0119] As an example, the input data may include operation data representing information regarding the operation of the work machine 30 performing construction work. Since the resistance value applied to the work machine 30 varies depending on the target soil for the construction work, it is desirable for the operation data to include, as an example, information indicating the hydraulic power of the work machine 30.

[0120] In this case, the trained model LM1 used in the estimation phase is a trained model that has machine-learned the correlation between the photographed data, the detected data, and the action data and the judgment result of the soil characteristics. In this case, the information processing device 10, 10B, 10C (hereinafter simply referred to as "information processing device 10, etc.") executes a step of acquiring action data, and estimates the soil characteristics using output data obtained by inputting the photographed data, the detected data, and the action data to the trained model LM1. When using action data as input data, it is desirable to provide an action acquisition unit as the measurement device and a third acquisition unit for action data in the information processing device.

[0121] In the above-described embodiment, the input data does not have to include image capture data. In other words, the detection data may be used as an input to the trained model LM1.

[0122] [Additional Note 2] In each of the above-described embodiments, the information processing devices 10, 10B, and 10C perform spectrum analysis on data representing sound and / or vibration detected by the measuring device 20, and use the analysis result data indicating the analysis result as an input to the trained model LM1. The detection data used as an input to the trained model LM1 is not limited to data indicating the analysis result of the spectrum analysis. As an example, the information processing devices 10 and 10B may directly use the detection data representing sound and / or vibration received from the measuring device 20 as an input to the trained model LM1.

[0123] Further, the information processing devices 10, 10B, and 10C may, for example, use data obtained by processing or converting the detection data received from the measuring device 20 using another method as input to the learned model LM1. For example, the estimation unit 113 may generate image data of a waveform represented by the detection data received from the measuring device 20, and use the generated image data as input data to the learned model LM1. In other words, the detection data used as input data to the learned model LM1 may be data representing an image of a waveform indicating at least one of sound and vibration generated by construction work. For example, the image data representing the waveform is time-series data of the sound waveform shown in FIG. 6.

[0124] The detection data to be input to the trained model LM1 may be image data representing the result of spectrum analysis of data representing a time series of sound and / or vibration. An example of the image data representing the result of spectrum analysis is the image data representing the result of spectrum analysis shown in FIG.

[0125] In this case, the second acquisition unit 112 performs a spectral analysis on the detection data received from the measurement device 20, and generates image data indicating an image of the spectral analysis result. The estimation unit 113 uses the image data generated by the second acquisition unit 112 as an input of the trained model LM1.

[0126] In this case, the detection data contained in the training data used to construct the trained model LM1 is, like the detection data contained in the input data, data in the form of an image representing the results of spectral analysis performed on data representing a time series of sound and / or vibration.

[0127] [Additional Note 3] In each of the above embodiments, the work machine 30 that performs construction work is not limited to the one exemplified in Fig. 1. The work machine 30 may be another machine that performs construction work.

[0128] [Additional Note 4] The functions of the information processing device 10 and the like according to each of the above-mentioned embodiments may be realized by a single device, or may be realized by a system in which a plurality of devices cooperate with each other. For example, a first device implementing the estimation phase execution unit 110 and a second device implementing the learning phase execution unit 120 may be configured as separate devices, and the above-mentioned information processing device 10 and the like may be realized by the first device and the second device. In this case, the second device may, for example, execute a step of acquiring first data including photographed data representing an image of the ground that is the target of the construction work, and detection data representing at least one of sound and vibration generated by the construction work, and second data indicating the soil characteristics of the ground, and a step of generating a trained model in which the correlation between the first data and the second data is machine-learned by using the first data and the second data corresponding to the first data. The first device and the second device are connected by a communication unit 12.

[0129] The functions of the measuring device 20 according to each of the above-mentioned embodiments may be realized by a single device, or may be realized by cooperation of multiple devices. For example, a first device that photographs the ground and generates photographed data and a second device that collects sound and generates detection data may be configured as separate devices, and the above-mentioned measuring device 20 may be realized by the first device and the second device. In this case, the information processing device 10 etc. receives the photographed data from the first device and receives the detection data from the second device.

[0130] In the above-described embodiment, the information processing device 10 or the like performs a spectral analysis such as FFT on the detection data and converts it into spectral data, but the device that performs the spectral analysis is not limited to the information processing device 10 or the like. A device other than the information processing device 10 or the like (e.g., the measurement device 20) may perform a spectral analysis on the detection data and convert it into spectral data indicating the analysis results (or image data indicating the analysis results). In this case, the device that performed the spectral analysis transmits the converted spectral data to the information processing device 10 via the communication path N1 or the like. The information processing device 10 or the like receives spectral data from another device.

[0131] As an example, the measurement device 20 may perform spectrum analysis on the detection data recorded by the detection unit 24 and generate data indicating the spectrum analysis result. In this case, the measurement device 20 transmits the generated spectrum result data to the information processing device 10. The information processing device 10 receives the spectrum result data from the measurement device 20 and uses the received spectrum result data as an input of the trained model LM1.

[0132] Furthermore, the configuration of the soil property estimation system 1 according to the present specification is not limited to that shown in the above-mentioned embodiment. As an example, the measuring device 20 according to the above-mentioned embodiment and the information processing device 10 may be configured as an integrated device. As another example, the information processing device 10 may be a cloud server connected to the measuring device 20 via a communication network such as the Internet.

[0133] [Additional Note 5] In the above embodiment, the measuring device 20 assigns shooting time information to the shooting data and also assigns detection time information to the detection data, and the information processing device 10 etc. associates the shooting data with the detection data based on the shooting time information assigned to the shooting data and the detection time information assigned to the detection data. The method of associating the shooting data with the detection data is not limited to the method shown in the above embodiment. As an example, the shooting data and the detection data may be associated by the following method. The measuring device 20 transmits the shooting data to the information processing device 10 etc. in real time every time it generates the shooting data, and transmits the detection data to the information processing device 10 etc. in real time every time it generates the detection data. The information processing device 10 etc. associates the shooting data with the detection data based on the timing of receiving the shooting data and the timing of receiving the detection data.

[0134] [Additional Note 6] The trained model LM1 used in each of the above-mentioned embodiments is not limited to those shown in the above-mentioned embodiments. The trained model LM1 may be, for example, a multi-timescale RNN (MTRNN), a recurrent neural network (RNN) such as a long short-term memory (LSTM), an autoregressive, integrated and moving average (ARIMA) model, or the like.

[0135] [Additional Note 7] In each of the above-mentioned embodiments, the information processing device 10, etc. may extract a part of the photographed data received from the measuring device 20 and use it as input of the trained model LM1. When the photographing unit 23 is fixed to the work machine 30, components such as a bucket, blade, ripper, and iron wheels are included in the photographed image, so the information processing device 10, etc. may generate photographed data in which these components are removed from the photographed image, and use the generated photographed data as input data of the trained model LM1. In other words, the information processing device 10, etc. may extract an image of an area in which the components of the work machine 30 are not included from the image photographed by the photographing unit 23, and use the photographed data representing the extracted image as input data of the trained model LM1. In this case, the photographed data included in the teacher data used to construct the trained model LM1 is data representing an image in which a part of the area is extracted from the image photographed by the photographing unit 23, similar to the photographed data included in the input data. In addition, in the extraction from the image, an algorithm such as semantic segmentation may be used to input the data to a learning model constructed by machine learning.

[0136] [Additional Note 8] In the above embodiment, if the work machine 30 has a movable part that moves during construction work, the measuring device 20 may control the shooting direction by following the movement of the movable part. As an example, if the work machine 30 is a backhoe, the measuring device 20 may identify the position of the claw of the backhoe and track and photograph the identified claw. In this case, the measuring device 20 controls the shooting direction of the photographing unit 23 so that the claw of the backhoe is included in the shooting range of the photographing unit 23. The position of the claw of the backhoe may be identified, for example, by the measuring device 20 executing image analysis processing on the image photographed by the photographing unit 23. Also, for example, a marker may be attached to the claw of the backhoe, and the measuring device 20 may detect the marker to identify the position of the claw.

[0137] [Software implementation example] The control blocks of the information processing devices 10, 10B, and 10C (particularly at least one of the first acquisition unit 111, the second acquisition unit 112, the estimation unit 113, the teacher data acquisition unit 121, and the generation unit 122) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0138] In the latter case, the information processing devices 10, 10B, and 10C are provided with a computer that executes instructions of a program, which is software that realizes each function. The computer includes, for example, one or more processors, and a computer-readable recording medium that stores the program. The object of the present invention is achieved by the processor reading the program from the recording medium and executing it in the computer. The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transient tangible medium," such as a ROM (Read Only Memory), as well as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like. The computer may further include a RAM (Random Access Memory) that expands the program. The program may be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) that can transmit the program. Note that one aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0139] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0140] 1, 1B, 1C Soil property estimation system 10, 10B, 10C Information processing device 11, 11B, 21 Control section 12, 22 Communications Department 13, 26 Storage section 20 Measuring Equipment 21 Control section 22 Communications Department 23 Photography Department 24 Detection unit 25 light source 26 Memory section 30, 30B working machine 110 Estimation Phase Execution Unit 111 First acquisition part 112 Second Acquisition Department 113 Estimation Department 120 Learning Phase Execution Unit 121 Teacher Data Acquisition Department 122 Generation part

Claims

1. acquiring photographic data representing an image of the ground that is the target of construction work; obtaining detection data representative of at least one of sounds and vibrations generated by the construction operations; a step of inputting data representing photographed images of the ground and data representing at least one of sounds and vibrations generated by construction work, outputting data representing at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground, and estimating at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground that is the subject of the work, using the output data obtained by inputting the photographed data and the detection data into a trained model that has been machine-learned to learn the correlation between the input data and the output data; The soil property estimation method is executed by an information processing device.

2. The information processing device further comprises: generating data representing a three-dimensional structural diagram of soil and rock layers in the ground that is the work target based on output data obtained by inputting the photographing data and the detection data into the trained model and position information corresponding to the photographing data and the detection data; The method for estimating soil properties according to claim 1, further comprising the steps of:

3. The detection data is data showing a result of spectrum analysis of time-series data showing at least one of the sound and the vibration generated by the construction work. The method for estimating soil properties according to claim 1 or 2.

4. The detection data is image data representing a result of spectrum analysis of the time series data. The method for estimating soil properties according to claim 3.

5. The step of acquiring the detection data includes a step of removing a predetermined frequency band of the sound or vibration recorded by a recording device during the construction work, and acquiring the data from which the predetermined frequency band has been removed as the detection data. The method for estimating soil properties according to any one of claims 1 to 4.

6. In the removing step, a frequency band corresponding to an operation sound or an operation vibration of a work machine performing the construction work is set as a frequency band to be removed. The method for estimating soil properties according to claim 5.

7. acquiring operation data representing information relating to the operation of a work machine performing the construction work; In the estimating step, the photographing data, the detection data, and the motion data are input to the trained model. The method for estimating soil properties according to any one of claims 1 to 6.

8. acquiring first data including photographed data representing images of the ground that is the target of the construction work and detection data representing at least one of sounds and vibrations generated by the construction work, and second data representing at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground; A step of generating the trained model by machine learning the correlation between the first data and the second data, using the first data as input data and the second data as output data; The method for estimating soil properties according to claim 1 , further comprising:

9. In the step of acquiring the first data and the second data, Obtaining shooting time information indicating a shooting timing of the shooting data and shooting position information indicating a shooting position corresponding to the shooting data, and obtaining detection time information indicating a detection timing of the detection data and detection position information indicating a detection position corresponding to the detection data; Correlating the photographing data with the detection data based on the photographing time information and the detection time information, and the photographing position information and the detection position information. The method for estimating soil properties according to claim 8.

10. acquiring first data including photographed data representing images of the ground that is the target of the construction work and detection data representing at least one of sounds and vibrations generated by the construction work, and second data representing at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground; A step of generating a trained model by machine learning a correlation between the first data and the second data, using the first data as input data and the second data as output data; The trained model generation method is executed by an information processing device.

11. a first acquisition unit that acquires photographed data representing an image of a ground that is a target of construction work; A second acquisition unit that acquires detection data representing at least one of noise and vibration generated by the construction work; an estimation unit that receives as input data data representing photographed images of the ground and data representing at least one of sounds and vibrations generated by construction work, and receives as output data data representing at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground, and estimates at least one of the rock type, degree of weathering, degree of unevenness, presence or absence of cracks, presence or absence of moisture, and amount of moisture contained in the ground that is the object of the work, using the output data obtained by inputting the photographed data and the detection data into a trained model that has been machine-learned to learn the correlation between the input data and the output data; A soil property estimation device comprising:

12. The estimation unit further comprises: Based on output data obtained by inputting the photographing data and the detection data into the trained model and position information corresponding to the photographing data and the detection data, data representing a three-dimensional structural diagram of the soil layer and rock layer in the ground that is the work target is generated. The soil property estimation device according to claim 11.

13. A program for causing a computer to function as the soil property estimation device according to claim 11, the program causing a computer to function as the first acquisition unit, the second acquisition unit, and the estimation unit.

Citation Information

Patent Citations

  • Forming method of abraded tire model, computer program for formation of abraded tire model, performance prediction method of abraded tire and abraded tire model

    JP2005263070A

  • Surface information analysis system and surface information analysis method

    JP2017220058A

  • Survey method of ground property

    JP2019167751A

  • Geological evaluation system, geological evaluation method, and geological evaluation program

    JP2020056254A

  • Construction machine with learning function

    WO2020241419A1