Information processing device, method, and program

Abstract

To provide a system that enables the reduction of omissions in a hammering inspection.SOLUTION: An information processing device has: first acquisition means for acquiring an instruction position indicated by a first user; second acquisition means for acquiring a hammering position where an object has been struck by a second user different from the first user; and determination means for determining whether the second user has struck the instruction position on the basis of the instruction position and the hammering position.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device that assists in performing a hammering inspection on specified hammering inspection points without omission, based on image data. [Background technology]

[0002] As a technique for inspecting defects on the walls of structures such as tunnels and bridges, inspections are performed by detecting abnormalities using images of the walls of structures such as tunnels and bridges, and tapping tests are performed using a tapping rod or hammer. This is a common method of checking whether there are any abnormalities such as cracks or floats inside the concrete. It is known that tapping tests require high skill in selecting the tapping location and judging the test results.

[0003] There is a structure inspection system described in Patent Document 1. This system discloses a technology for performing a hammering inspection using a flying robot with a percussion rod, hammer, etc., according to the inspection results obtained by detecting abnormalities in images of the wall surface of a structure, in order to reduce the inspection time for the hammering inspection. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2020-079526 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional techniques disclosed in the above-mentioned patent documents, the surroundings of tunnels and bridges are not necessarily suitable for flying a flying robot such as a radio-controlled helicopter, and in some cases are even unsuitable for flying. In addition, there is a concern that the propeller and motor noise of the flying robot may affect the hammering test.

[0006] On the other hand, when conducting hammering inspections manually, it is necessary to have a high level of skill to select the necessary hammering locations. If it is not feasible to send a select number of highly skilled people to all sites, a system is needed that allows highly skilled people to accurately perform hammering inspections at pre-selected points based on the location of deformation.

[0007] Therefore, an object of the present invention is to provide a system that makes it possible to reduce omissions in hammering inspection. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides A first acquisition means for acquiring an indication position indicated by a first user; A second acquisition means for acquiring a hitting position where an object is hit by a second user different from the first user; and a determination means for determining whether the second user has struck the designated position based on the designated position and the striking sound position. It is characterized by: Effect of the Invention

[0009] According to the present invention, it is possible to reduce oversights in hammering inspection. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the configuration of a system according to an embodiment of the present invention. [Diagram 2] A flowchart outlining the photography process for image-based defect detection inspections. [Diagram 3] A flowchart showing an overview of the work of an operator who selects a hammering location using the hammering inspection system according to an embodiment of the present invention. [Figure 4] A display example of a UI displayed on an information processing device when selecting a hammering location using the hammering inspection system according to an embodiment of the present invention. [Diagram 5]A flowchart showing an overview of the work of an operator who performs a hammering test using the hammering test system according to an embodiment of the present invention. [Figure 6] FIG. 1 shows a hitting sound range according to an embodiment of the present invention. [Figure 7] FIG. 1 is a diagram showing an imaging range according to an embodiment of the present invention; [Figure 8] FIG. 2 is a diagram showing a display unit according to an embodiment of the present invention; [Figure 9] FIG. 1 is a diagram showing an imaging range according to an embodiment of the present invention; [Figure 10] FIG. 1 shows a live view display according to an embodiment of the present invention. [Figure 11] FIG. 1 is a diagram showing the entire image data according to an embodiment of the present invention; [Figure 12] FIG. 2 is a diagram showing a display example of a display unit according to an embodiment of the present invention; [Figure 13] A flowchart showing an overview of the work performed when an operator who performs a hammering test using the hammering test system according to an embodiment of the present invention adds a hammering location. [Figure 14] FIG. 1 is an image showing the display of the inspection results in the hammering inspection system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which: Fig. 1 is a block diagram of an imaging system according to an embodiment of the present invention.

[0012] Reference numeral 100 denotes an information processing device constituting the hammering inspection system. Reference numeral 101 denotes a system control unit that performs system control of the information processing device. Reference numeral 102 denotes an image processing unit that performs processing such as abnormality detection by image analysis of input image data, feature point extraction, comparison, etc., and outputs the processing results. Reference numeral 103 denotes an internal memory provided in the image processing device 110, and is used to hold input / output data and temporary data of the image processing unit 102, and also to hold temporary data during display and communication processing. Reference numeral 104 denotes a display unit that includes a TFT liquid crystal panel or an organic EL panel, and displays output data of the image processing unit 102. It also doubles as a user input unit with a touch panel configuration. Reference numeral 105 denotes a communication unit that connects to an external system of the information processing device 100 by wireless or wired connection, and transmits and receives data. Reference numeral 106 denotes a microphone that collects sounds around the information processing device 100 to be recorded as sounds. Reference numeral 107 denotes a recording medium such as a memory card for recording images of structures such as tunnels and bridges that have been photographed, and the tapping test positions as positions relative to the images, and is composed of a semiconductor memory, an optical disk, a magnetic disk, or the like.

[0013] Reference numeral 110 denotes an imaging device constituting the hammering inspection system. Reference numeral 111 denotes a system control unit that performs system control of the imaging device 110, and performs control related to imaging, image processing, communication, and the like. Reference numeral 112 denotes an imaging unit that performs optical systems for exposing and imaging the imaging sensor, imaging sensor, AD conversion, gain control, and correction control. Reference numeral 113 denotes an image processing unit that performs processes such as correction, development, compression, and encoding on the data captured by the imaging unit 112 to generate image data. It also performs control and processing for improving the quality of captured images such as AF, AE, WB, image stabilization, and strobe dimming, as well as image analysis, detection, and recognition. Reference numeral 114 denotes an internal memory provided in the imaging device 110 and a removable external memory. The internal memory is used when temporarily storing data when the imaging unit 112 acquires imaging data or when the image processing unit 113 processes images, and when temporarily storing data when performing communication processing. Reference numeral 115 denotes a communication unit that connects to an external system of the imaging device 110 by wireless or wired connection, and transmits and receives data.

[0014] The communication unit 115 of the information processing device 100 is connected to the communication unit 106 of the camera 100 by a wireless or wired communication method. A live view image is captured by the imaging unit 112 of the imaging device and generated by the image processing unit 113. The image is transferred to the wirelessly connected information processing device via the communication unit 115, the live view image is received by the communication unit 105 of the information processing device, and the image stored in the memory 103 is displayed on the display unit 104. In this way, the information processing device 100 also functions as an electronic viewfinder and can perform a live view display.

[0015] FIG. 2 is a flow chart showing an outline of the photographing work of the image of the structure for the inspection by the abnormality detection. It is assumed that the preparation for the photographing work, such as grasping the shape and size of the inspection object and the photographing plan, has already been done. In S201, the photographing area of ​​the structure is divided and photographed. It is difficult to realize the photographing of the structure such as a bridge pier or a tunnel with a generally available resolution camera with a high image quality that can perform a high-precision inspection for damage and other deterioration with a single photograph. Therefore, many divided photographs are taken and the whole photographing area is photographed. In S202, the divided images are synthesized. Whether to use a synthesized image or a divided image for the image analysis of the abnormality such as cracks and corrosion depends on the image analysis sequence or algorithm. The synthesis is performed to confirm whether the photographing area of ​​the photographing object has been performed without failure or omission, and to manage and display the position of the abnormality such as cracks and corrosion recorded in the image in the structure. In the hammering inspection system according to the embodiment of the present invention, it is assumed that the synthesized image of the structure for the inspection by the abnormality detection is already prepared.

[0016] A hammering inspection system according to an embodiment of the present invention will be described with reference to FIGS.

[0017] 3 is a flow chart showing an outline of the work of an operator who selects a tapping location using the tapping inspection system according to an embodiment of the present invention. The operator's work and the processing of the information processing device according to the present invention are expressed by different notation columns. Numbers S301 to 303 represent the workflow of the operator, and S310 to 312 represent the processing of the information processing device 100 according to the present invention.

[0018] In S301, the worker instructs the information processing device 100 to display a composite image of a structure for inspection by abnormality detection (S310).

[0019] In S310, the system control unit 101 reads out the composite image of the structure from the recording medium 107, and the image processing unit 102 performs anomaly detection from the composite image of the structure by image analysis, and displays the composite image of the structure and the anomaly detection results superimposed on the display unit 104.

[0020] In S302, the worker designates a position (hammering inspection point) at which the hammering inspection is to be performed based on the position of the abnormality by touching the information processing device 100 (S311).

[0021] In S311, the system control unit 101 displays on the display unit 104 that the instruction has been received, and stores in the memory 103 the relative position of the instructed position with respect to the composite image of the structure.

[0022] In S303, the worker instructs the information processing device 100 to store the position where the hammering inspection is to be performed (S312).

[0023] In S312, the system control unit 101 stores in the recording medium 107 the relative position of the instructed position with respect to the composite image of the structure, which has been temporarily stored in the memory 103.

[0024] FIG. 4 is a display example of a UI displayed on the information processing device 100 to supplement the explanation of FIG. 3. 401 and 402 in FIG. 4 superimpose a composite image of a structure and a result of the abnormality detection. While 402 displays the entire composite image of the structure, 401 accepts pinch-in / pinch-out and drag operations so that the operator can easily check the abnormality and the result of the abnormality detection. This allows the operator to enlarge / reduce the superimposed display of the composite image of the structure and the result of the abnormality detection, and to move the enlarged position. 403 displays information related to the hammering inspection, such as the date of creation of this inspection instruction, the location where the hammering inspection is to be performed, the number of designated hammering inspection points, and the planned man-hours for the hammering inspection derived by multiplying the number of hammering inspection points by a certain coefficient. 404 to 406 display the designated hammering inspection points. When the enlargeable composite image 401 of the structure is touched by a touch operation 408, a hammering inspection point is designated. The relative position of the instructed position with respect to the composite image of the structure is stored in the memory 103, and the hammering inspection point display 404 is displayed. 405 to 407 show hammering inspection point displays according to the relative positions of the hammering inspection points stored in the memory 103 with respect to the composite image of the structure in the entire composite image of the structure. The display of the hammering inspection point 404 on the display of the entire composite image of the structure 402 is 405. In order to realize both the ability to enlarge and check the deformation and specify the hammering inspection point, and the ability to check the specified hammering inspection points in a list, the display of the enlargeable composite image of the structure 401 and the display of the entire composite image of the structure 402 are separated. When the inspection point specification completion button 409 is touched by the touch operation 410, the relative position of the instructed position with respect to the composite image of the structure, which was temporarily stored in the memory 103, is stored in the recording medium 107.

[0025] 5 is a flow chart showing an outline of the work of an operator who performs a hammering inspection using a hammering inspection system according to an embodiment of the present invention. The operator's work and the processing of the information processing device according to the present invention are expressed using different notation columns. Numbers S5001 to 5008 represent the workflow of the operator, S5100 to 5115 represent the processing of the information processing device 100 according to the present invention, and S5201 to 5202 represent the processing of the information processing device 100 according to the present invention.

[0026] FIG. 6 is an image diagram to supplement the explanation of FIG. 5. In FIG. 6(a), 600 represents an infrastructure structure. The surface 601 is the area to be subjected to the hammering test. 602 represents a defect such as a crack found in the infrastructure structure 600, and is also a feature point that can be detected on the surface of a structure made of uniform concrete or the like. FIG. 6(b) represents a composite image of a structure for inspection by defect detection, created by S201-202, and 603 represents that a hammering test point has been specified at the corresponding position by S310-312.

[0027] In S5001, the operator instructs the information processing device 100 to start a hammering inspection position designation mode (S5100).

[0028] In S5100, the system control unit 101 reads out the composite image of the structure from the recording medium 107, and reads out the first hammering inspection point, which is a relative position with respect to the composite image of the structure.

[0029] In S5101, the system control unit 101 instructs the image capturing device 110 via the communication unit 105 to supply a live view image (S5201).

[0030] In S5201, the system control unit 111 receives a live view supply instruction from the information processing device 100 via the communication unit 115, and starts generating a live view image using the imaging unit 112 and image processing unit 113.

[0031] In S5002, the worker directs the imaging device 110 toward the inspection object to search for the hammering inspection position (S5202).

[0032] In S5202, the system control unit 111 generates live views one by one and transmits image data from the communication unit 115 (S5102).

[0033] In S5102, the system control unit 101 sequentially receives image data from the imaging device 110 via the communication unit 105 (S5103).

[0034] In S5103, the system control unit 101 performs a search for matching of feature points of the deformation in the composite image of the structure and the sequential image data (S5104).

[0035] In S5104, the system control unit 101 calculates relative position information of the sequential image data received in S5102 with respect to the designated position for the hammering test (S5105).

[0036] FIG. 7 is an image diagram to supplement the explanation of FIG. 5. FIG. 7(a) shows an infrastructure structure and the imaging range of the imaging device 110 in S5002. 701 in FIG. 7(a) shows the same infrastructure structure as 601 in FIG. 6(a), and the dotted frame of 702 shows the imaging range of the imaging device 110. FIG. 7(b) shows an image diagram of the feature point match search with the composite image of the structure in S5103. 711 shows the composite image of the structure, and 712 shows the sequential image data transferred from the imaging device 110 at this time. 713 shows feature point information included in the composite image of the wall surface stored in the memory 103, and 714 shows the feature point search result in the image data 612. In S5103 in FIG. 5, a match search is performed between the feature point information 713 and the feature point search result 714, and based on the coordinate information included in the feature point information 713, it is specified which position of the image data 712 corresponds to in the coordinates on the composite image of the structure. Fig. 7(c) is an image diagram showing relative position information of image data 712 with respect to a hammering inspection point on a composite image 711 of a structure. 721 represents the shooting range of image data 712 on the composite image of the structure. 722 represents one of the hammering inspection points specified by the worker who selects the hammering location in S311 of Fig. 3. 723 represents relative position information of 722 with 721 as the reference.

[0037] In S5105, the system control unit 101 judges whether the position on the composite image of the wall surface received in S5102 includes the designated position for the hammering test in the sequential image data based on the relative position information with respect to the coordinate information on the composite image of the wall surface generated in S5104. If it does not include the designated position, the process proceeds to (S5106), and if it does include the designated position, the process proceeds to (S5107).

[0038] In S5106, the system control unit 101 generates vector information as position related information and guide information indicating the direction of the designated position for the hammering test based on the vector information, and sequentially displays it on the display unit 104 together with the image data.

[0039] In S5003, the worker can adjust the direction of the imaging device 110 toward the structure and change the angle of view according to the guide information, which is displayed on the display unit 104 of the information processing device 100 and indicates the direction of the designated position for the hammering inspection and the sequential image data (S5002).

[0040] Fig. 8 is a diagram to supplement the explanation of Fig. 5. Reference numeral 801 denotes the display unit 104 of the information processing device 100. Reference numeral 802 denotes a live view display that displays image data acquired sequentially, and reference numeral 803 denotes guide display information. In S5106, the guide display information is displayed on the display unit 104 of the information processing device 100 so as to be superimposed on the sequential image information, as shown in Fig. 8.

[0041] In S5107, the system control unit 101 generates guide information indicating the designated positions for the hammering test as position related information, and sequentially displays it on the display unit 104 together with the image data.

[0042] In S5004, the worker can perform a hammering inspection of the structure according to the sequential image data and guide information indicating designated positions for the hammering inspection displayed on the display unit 104 of the information processing device 100 (S5108).

[0043] FIG. 9 is a diagram to supplement the explanation of FIG. 5. FIG. 9(a) shows an infrastructure structure and the imaging range of the imaging device 110 in S5002, similar to FIG. 7(a). 901 indicates the same infrastructure structure as 601 in FIG. 6(a), and the dotted frame of 902 indicates the imaging range of the sequential image data transferred from the imaging device 110 at this time. FIG. 9(b) is an image diagram of the feature point matching search with the composite image of the structure in S5103, similar to FIG. 7(b). 911 indicates the composite image of the structure, and 912 indicates the sequential image data transferred from the imaging device 110 at this time. 913 indicates feature point information included in the composite image range of the structure stored in the memory 103, and 914 indicates the feature point search result in the image data 912. FIG. 9(c) is an image diagram showing the relative position information of the image data 712 with respect to the hammering inspection point on the composite image 911 of the structure. 921 indicates the imaging range of the image data 912 on the composite image of the structure. Reference numeral 922 denotes one of the hammering inspection points designated by the worker who selects the hammering location in S311 of Fig. 3. It can be determined that the shooting range 921 of the image data 912 on the composite image 911 of the structure includes the hammering inspection point 922 designated by the worker who selects the hammering location.

[0044] FIG. 10 is a diagram to supplement the explanation of FIG. 5. 1001 indicates the display unit 104 of the information processing device 100. 1002 indicates a live view display that displays image data acquired sequentially. 1003 indicates a display showing one of the hammering inspection points designated by the worker who selects the hammering location in S311 of FIG. 3 as guide display information. 1004 indicates a display showing the feature point search result 914 or 913 included in the shooting range of the image data 912, which is further displayed as guide display information. The guide display information 1003 may be another display showing one of the hammering inspection points designated by the worker who selects the hammering location in S311 of FIG. 3, or may be a display of a character string, an icon, a frame, or a combination thereof. The guide display information 1004 may be a display showing the shape of the abnormality such as a crack detected as a feature point as it is, or may be another display method such as a rectangle.

[0045] In S5108, the system control unit 101 detects the timing at which the worker made the tapping sound in S5004 based on the microphone output and the voice frequency, and stores in the memory 103 the tapping sound data and image data for a certain period of time at the detected timing.

[0046] In S5109, the system control unit 101 identifies the relative position of the hitting sound position in the composite image of the structure by analyzing the image data acquired in S5107.

[0047] FIG. 11 is a diagram to supplement the explanation of FIG. 5. 1101 represents the entire image data acquired in S5107. 1102 represents a hammer used for the impact inspection. 1103 represents the impact point when the impact is performed with the hammer 1102. 1104 represents a defect such as a crack found in an infrastructure structure near the impact point, and is also a feature point detectable on the surface of a structure made of uniform concrete or the like. 1105 represents the entire composite image of the structure. Based on the feature point information included in the composite image range of the structure stored in the memory 103 and the feature point of the defect 1104, a position 1106 of the defect 1104 near the impact point 1103 in the composite image of the structure is specified. This makes it possible to specify the relative position of the impact point 1103 in the composite image of the structure.

[0048] In S5110, the system control unit 101 judges whether the distance between the hammering inspection point designated by the worker who selects the hammering location and the hammering position obtained in S5109 is within a threshold value. If it is not included, proceed to (S5111), and if it is included, proceed to (S5112).

[0049] In S5111, the system control unit 101 displays on the display unit 104 guide information indicating that the striking position is different (S5005).

[0050] In S5004, the worker tries to make a tapping sound again in accordance with the guide information displayed on the display unit 104 of the information processing device 100, indicating that the tapping position is different (S5002).

[0051] In S5112, the system control unit 101 stores the hitting sound data held in the memory 103 in the recording medium 107 in association with the relative position in the composite image of the structure, and displays guide information to that effect on the display unit 104. In order to distinguish the designated hitting sound data from hitting sound data at an arbitrary position, which will be described later with reference to Fig. 13, the hitting sound data is stored as file information indicating that the hitting sound data is at a designated position (S5113).

[0052] In S5113, the system control unit 101 judges whether or not the inspection of all hammering inspection positions designated by the operator who selects the hammering locations has been completed. If not completed, the process proceeds to (S5114), and if completed, the process proceeds to (S5115).

[0053] In S5114, the system control unit 101 displays on the display unit 104 guide information to the effect that the next specified tapping test position will be read and tested (S5006).

[0054] In S5006, the worker attempts to perform tapping again in accordance with the guide information displayed on the display unit 104 of the information processing device 100, which indicates that an inspection should be performed at the next specified tapping inspection position (S5002).

[0055] In S5115, the system control unit 101 displays on the display unit 104 guide information indicating that the inspection of all the designated hammering inspection positions has been completed (S5006).

[0056] In S5007, the worker follows the guide information displayed on the display unit 104 of the information processing device 100 to the effect that inspection of all the designated hammering inspection positions has been completed, and ends the work.

[0057] Fig. 12 is a diagram to supplement the explanation of Fig. 5. 1201 to 1204 are display examples of the display unit 104 of the information processing device 100. 1201 is a display example of S5111. 1202 is a display example of S5112. 1203 is a display example of S5114. 1204 is a display example of S5115.

[0058] It may be possible to perform a hammering test not only at hammering test positions designated by an operator who selects hammering locations, but also at hammering test positions that concern the operator who performs the hammering test.

[0059] 13 is a flow chart showing an outline of the work of performing a hammering test at a hammering test position of concern to an operator performing a hammering test according to an embodiment of the present invention. The operator's work and the processing of the information processing device according to the present invention are expressed using different notation columns. Numbers S13001-13003 represent the workflow of the operator, S13101-13105 represent the processing of the information processing device 100 according to the present invention, and S13201-13202 represent the processing of the information processing device 100 according to the present invention.

[0060] In S13001, the operator instructs the information processing device 100 to start the hammering test position arbitrary mode (S13101).

[0061] In S13101, the system control unit 101 instructs the image capturing device 110 via the communication unit 105 to supply a live view image (S13201).

[0062] In S13201, the system control unit 111 receives a live view supply instruction from the information processing device 100 via the communication unit 115, and starts generating a live view image using the imaging unit 112 and image processing unit 113.

[0063] In S13202, the system control unit 111 generates live views one by one and transmits image data from the communication unit 115 (S13102).

[0064] In S13102, the system control unit 101 sequentially receives image data from the imaging device 110 via the communication unit 105, and performs live view display on the display unit 104 (S13002).

[0065] In S13002, a worker performs a tapping test on an arbitrary position of the structure (S13103).

[0066] In S13103, the system control unit 101 detects the timing when the worker performed the tapping sound in S13002 from the output of the microphone, and stores in the memory 103 the tapping sound data and image data for a certain period of time at the detected timing.

[0067] In S13104, the system control unit 101 identifies the relative position of the hitting sound position in the composite image of the structure by analyzing the image data acquired in S13103. The explanation is omitted since it is the same as in FIG.

[0068] In S13105, the system control unit 101 associates the hitting sound data held in the memory 103 with the relative position in the composite image of the structure, stores it in the recording medium 107, and displays guide information to that effect on the display unit 104 (S13003). In order to distinguish between the designated hitting sound data and hitting sound data at an arbitrary position, the information indicating that it is hitting sound data at an arbitrary position is stored as file information. The explanation of the guide information is omitted since it is the same as 1202.

[0069] In S13003, the worker determines whether to end the work. If not, the process proceeds to (S13002).

[0070] Fig. 14 is an image diagram showing the display of the inspection results in the hammering inspection system according to the embodiment of the present invention. When the inspection results are displayed, first, a display like 1401 in Fig. 14(a) is performed. 1402 to 1404 are displayed according to the relative position of the structure in the composite image of the hammering sound data. The difference between 1402, 1403 and 1404 is the display according to the difference in file information, whether it is specified hammering sound data or hammering sound data at an arbitrary position.

[0071] Numeral 1405 indicates that the position 1402 where the hammering sound data is located is touched on 1401, which transitions to the display in FIG. 14(b) and also plays back the hammering sound data. Numeral 1406 indicates information about nearby abnormalities. Numeral 1407 displays information about the hammering sound data. The worker who selected the hammering sound location, the worker who performed the hammering sound inspection, the hammering sound inspection results, and the hammering sound inspection location are displayed. Numeral 1408 is an example of displaying image data acquired in S5107 or S13103. The image data acquired in S5107 or S13103 is saved in association with the hammering sound data, and by displaying it when the hammering sound data is selected, the correctness of the position of the acquired hammering sound data at the time of hammering can be reconfirmed even when the inspection results are displayed.

[0072] 1409 indicates that a position on 1401 where there is no tapping sound data is touched, and the display transitions to that shown in Fig. 14(c). 1409 indicates information on nearby abnormalities. As shown in 1410, it is displayed that there is no information on tapping sound data.

[0073] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0074] <Configuration 1> A first acquisition means for acquiring an indication position indicated by a first user; A second acquisition means for acquiring a hitting position where an object is hit by a second user different from the first user; and a determination means for determining whether the second user has struck the designated position based on the designated position and the striking sound position. 23. An information processing apparatus comprising:

[0075] <Configuration 2> and a notification means for notifying the second user to retry the strike when the determination means has not determined that the second user has struck the designated position. 2. The information processing device according to configuration 1.

[0076] <Configuration 3> the first acquisition means acquires the designated position from an instruction means that accepts an instruction of the designated position from the first user; The indicating means receives the indicated position based on a position of the deformation of the object. 2. The information processing device according to configuration 1.

[0077] <Configuration 4> An image capturing means for capturing an image of the second user hitting the object; A detection means for detecting a shape of a deformation of the object based on the image; The indicated position is displayed based on the image and the shape of the deformation. 2. The information processing device according to configuration 1.

[0078] <Component 5> a recording means for recording a hitting sound of the second user hitting the target object when the second user hits the designated position; 2. The information processing device according to configuration 1.

[0079] <Method 1> a first acquisition step of acquiring a designated position designated by a first user; A second acquisition step of acquiring a hitting position where the object is hit by a second user different from the first user; and a determining step of determining whether the second user has struck the designated position based on the designated position and the striking sound position. 23. An information processing method comprising:

[0080] <Program 1> 6. A computer program for controlling each unit of the image processing system according to any one of configurations 1 to 5 by a computer. [Explanation of symbols]

[0081] 100 Information processing device 101 System control section 102 Image processing section 104 Display section 106 Mike 110 Imaging device 111 System control section 112 Imaging unit 113 Image Processing Unit

Claims

1. A first acquisition means for acquiring the indicated position for a tapping test instructed by a first user on a reference image obtained by photographing an object, A second acquisition means for acquiring the strike position by the second user on the reference image based on an image captured when a second user, different from the first user, strikes the object, The system includes a determination means for determining whether the second user has struck the indicated position based on the indicated position and the striking position. An information processing device characterized by the following:

2. The information processing apparatus according to Claim 1, characterized in that the determination means determines that the second user has struck the indicated position when the distance between the indicated position and the striking position on the reference image is within a predetermined threshold.

3. If the determination means does not determine that the second user has struck the indicated position, the system further includes a notification means for notifying the second user to repeat the strike. The information processing apparatus according to feature 1.

4. The first acquisition means acquires the indicated position based on the location or shape of the deformation of the object. The information processing apparatus according to feature 1.

5. The information processing apparatus according to Claim 1, wherein the second acquisition means identifies the position of the captured image on the reference image by comparing the deformation or characteristic points of the object included in the captured image with the deformation or characteristic points of the object included in the reference image, and acquires the striking position.

6. The invention further comprises a display means for displaying the captured image, The display means displays guide information indicating the indicated position superimposed on the captured image. The information processing apparatus according to feature 1.

7. The information processing apparatus according to claim 6, wherein the display means displays guide information indicating the direction of the indicated position when the indicated position is outside the imaging range of the captured image.

8. The information processing apparatus according to claim 1, characterized in that the second acquisition means acquires the impact position based on the captured image corresponding to the timing at which the impact sound by the second user is detected.

9. The information processing apparatus according to claim 1, further comprising recording means for recording the impact sound in association with the impact sound location on the reference image when the determination means determines that the second user has struck the indicated position.

10. The information processing apparatus according to claim 9, wherein the recording means records the impact sound in a manner that allows identification of whether it is an impact sound at the indicated position or an impact sound at a position arbitrarily selected by the second user.

11. The information processing apparatus according to claim 9, characterized in that the recording means records the impact sound and the captured image corresponding to the timing at which the impact sound was detected in association with each other.

12. The information processing apparatus according to claim 1, characterized in that the reference image is a composite image obtained by combining a plurality of images obtained by dividing and photographing the object.

13. A first acquisition step of acquiring the indicated position for a tapping test instructed by a first user on a reference image obtained by photographing an object, A second acquisition step involves acquiring the striking position by the second user on the reference image based on the captured image when a second user, different from the first user, strikes the object, The system includes a determination step of determining whether the second user struck the indicated position based on the indicated position and the striking position. An information processing method characterized by the following:

14. A computer program for controlling each means of the information processing apparatus described in any one of claims 1 to 12 by a computer.

Images