Investigation Support Device and Investigation Support Method

Abstract

To improve accuracy in research.SOLUTION: A research support device performs: first acquisition processing of acquiring a research result including at least one of text data, captured image data, and voice data in research at a research date; second acquisition processing of acquiring first captured image data captured from the front of a researcher at the research date and second captured image data captured from the back of the researcher at the research data; specification processing of specifying a research position at the research date on the basis of at least one of the first captured image data and the second captured image data acquired by the second acquisition processing; and associating processing of associating the research result acquired by the first acquisition processing, the second captured image data, and the research position specified by the specification processing with the research date.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an investigation support apparatus and an investigation support method for supporting investigations.

Background Art

[0002] The following Patent Document 1 discloses a behavior observation apparatus that records and reproduces with simple operations while an observer observes the behavior of a participant. This behavior observation apparatus displays a participant on a scene to be observed as an icon, displays the movement of the participant by a trajectory, arranges and displays recorded images or their thumbnails of the behavior that the participant should pay attention to in chronological order, and when moving the icon according to the movement of the participant, stores the trajectory in a storage device together with time data, displays a mark at an arbitrary point of interest in the observation screen or the recorded image, and requests input of attribute data including the meaning of the mark.

[0003] The following Patent Document 2 discloses an adapter unit that can be connected to an existing detector installed at a site, can cope with an increase in the content and amount of measurement management data and a change in the on-site measurement point accompanying the progress of construction, and can perform on-site deformation display and data collection of civil engineering and architectural measurement data. This adapter unit includes a detector interface unit, two transmission interface units for connecting adjacent units via a balanced transmission line cable and a power cable, an LED display unit for notifying on-site workers of the deformation of a structure, a signal conversion unit for converting an analog signal into a digital signal, a memory unit, an information processing unit that outputs a display signal to the LED display unit when the digital signal is out of the allowable range, and an identification number, displays the on-site deformation according to the data of the detector, and transmits data to a data collection device provided at the end of a plurality of connected units.

[0004] The following Patent Document 3 discloses a method for downloading security investigation items. This method reads a two-dimensional code attached to a supply facility and a consumer facility main body of LP gas, downloads an investigation form (investigation items) related to each security operation to a mobile terminal in real time at the work site, and aggregates the investigation results to a server by data communication.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the range that the investigator can see is only within the viewing angle, and the video information outside the viewing angle cannot be seen, and there may be omissions in the investigation. The elimination of such omissions is not considered in the above - described conventional technologies.

[0007] The present invention aims to improve the investigation accuracy.

Means for Solving the Problems

[0008] An investigation support device which is an aspect of the invention disclosed in the present application is an investigation support device having a processor that executes a program and a storage device that stores the program. It is possible to access a database that stores the feature information and location information of an object, The processor executes a first acquisition process of acquiring an investigation result including at least one of text data, imaging data, and audio data in the investigation at the investigation date and time, a second acquisition process of acquiring first imaging data captured from the front of the investigator at the investigation date and time and second imaging data captured from the back of the investigator at the investigation date and time, a specifying process of specifying the investigation position at the investigation date and time based on at least one of the first imaging data and the second imaging data acquired by the second acquisition process, and an associating process of associating the investigation result acquired by the first acquisition process, the second imaging data, and the investigation position specified by the specifying process with the investigation date and time. and in the specific processing, when there is a second object with second feature information that matches the feature information of the first object in the database in the first imaging data, the processor determines the location information of the first object at the investigation location at the investigation date and time. When there are a plurality of second objects with second feature information that match the first feature information of the first object in the database in the first imaging data, and there is a fourth object with fourth feature information that matches the third feature information of the third object in the database in the second imaging data, based on the location information of the third object, the location information of a specific second object among the plurality of second objects is determined at the investigation location at the investigation date and time. characterized by this.

Effect of the Invention

[0009] According to a typical embodiment of the present invention, the investigation accuracy can be improved. Problems, configurations, and effects other than those described above will be clarified by the description of the following examples.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0011] <Example of System Configuration of Survey Support System> FIG. 1 is an explanatory diagram showing an example of a system configuration of a survey support system. The survey support system 100 includes a server 101, an omnidirectional camera 102, and a tablet 103. The server 101 is communicably connected to the omnidirectional camera 102 and the tablet 103. The server 101 can access a site map DB 110 and a survey log DB 120. Specifically, for example, the server 101 may hold the site map DB 110 and the survey log DB 120, or may be communicably connected to a computer that holds the site map DB 110 and the survey log DB 120.

[0012] The site map DB 110 stores map information of the site to be surveyed. The survey log DB stores, as a survey log, data acquired from the omnidirectional camera 102 and the tablet 103. The server 101 executes a process of assisting the survey by the surveyor 130 using the data acquired from the omnidirectional camera 102 and the tablet 103.

[0013] The omnidirectional camera 102 captures an object to generate omnidirectional imaging data. The omnidirectional camera 102 includes a front imaging unit 102A and a rear imaging unit 102B. The front imaging unit 102A and the rear imaging unit 102B each have an imaging element and a lens. The back of the imaging element of the front imaging unit 102A is disposed opposite to the back of the imaging element of the rear imaging unit 102B. The front imaging unit 102A and the rear imaging unit 102B each have the same viewing angle (for example, 210 degrees). The omnidirectional camera 102 is installed on the top of the investigator 130 such that the direction of the front imaging unit 102A (the optical axis of the lens) is the same as the line of sight direction of the investigator 130.

[0014] The front imaging unit 102A and the rear imaging unit 102B respectively generate front imaging data and rear imaging data. The omnidirectional camera 102 generates omnidirectional imaging data by stitching the front imaging data and the rear imaging data together. Note that the generation of the omnidirectional imaging data may be executed by the server 101 or the tablet 103.

[0015] The omnidirectional camera 102 has a microphone and can also record ambient sounds. The microphone may be provided in each of the front imaging unit 102A and the rear imaging unit 102B. By setting the direction of each microphone to the direction of the optical axis of each lens of the front imaging unit 102A and the rear imaging unit 102B, it is possible to generate front-side audio data together with the imaging data of the front imaging unit 102A, and it is possible to generate rear-side audio data together with the imaging data of the rear imaging unit 102B.

[0016] The omnidirectional camera 102 may be communicably connected to the tablet 103. In this case, the omnidirectional camera 102 can transmit the generated data to the server 101.

[0017] The tablet 103 is a computer carried by the investigator 130 and can execute an investigation support application. The investigation support application is software that can display investigation items about the investigation target and input investigation results about the investigation items by user operation. The investigation results are text data input by user operation, imaging data (still images or moving images) captured by the camera of the tablet by user operation, or audio data recorded by user operation. Also, the tablet 103 may have a positioning function for the current position using the GPS system. Note that the tablet 103 may have the functions of the above-described server 101.

[0018] <Hardware Configuration Example of the Investigation Support Device (Server 101, Tablet 103)> FIG. 2 is a block diagram showing a hardware configuration example of the investigation support device. The investigation support device 200 includes a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, the storage device 202, the input device 203, the output device 204, and the communication IF 205 are connected by a bus 206. The processor 201 controls the investigation support device 200. The storage device 202 serves as a working area for the processor 201. Also, the storage device 202 is a non-temporary or temporary recording medium that stores various programs and data. Examples of the storage device 202 include a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), and a flash memory. The input device 203 inputs data. Examples of the input device 203 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 204 outputs data. Examples of the output device 204 include a display, a printer, and a speaker. The communication IF 205 connects to a network and transmits and receives data.

[0019] <On-site Map DB 110> FIG. 3 is an explanatory diagram showing an example of the on-site map DB 110. The on-site map DB 110 stores object information arranged at the site to be investigated. The object information is created in advance. The on-site map DB 110 has, as fields, an object ID 301, an object name 302, shape information 303, and position information 304. The combination of the values of the fields 301 to 304 in the same row defines the object information that identifies the object at the site. In particular, the object ID 301 and the shape information 303 are regarded as the characteristic information of the object.

[0020] The object ID 301 is identification information that uniquely identifies an object. Different object IDs 301 are assigned to objects of the same type. The object name 302 is the name of the object identified by the object ID 301. The shape information 303 is data indicating the three-dimensional shape of the object identified by the object ID 301. The position information 304 is the position where the object identified by the object ID 301 is placed at the site, for example, the coordinate values in the local coordinate system of the site.

[0021] Note that if the site is an outdoor facility, the position information 304 may be the latitude and longitude information measured by the tablet 103 using the GPS system.

[0022] <Examples of the actions of the investigator 130> Figure 4 is a time chart showing examples of the actions of the investigator 130. The investigator 130 wears the omnidirectional camera 102 on the top of the head and carries the tablet 103 to investigate the site. In Figure 4, a certain investigation day X will be taken as an example for explanation.

[0023] The investigator 130 starts the investigation at the site from position P0 at time tx0. The omnidirectional camera 102 performs imaging and recording from the start time tx0 of the investigation by the operation of the investigator 130.

[0024] The investigator 130 conducts an investigation according to the investigation item INV1 at position P1 at time tx1. For example, the investigator 130 inspects the equipment at position P1, checks whether the work by the worker is appropriate, or interviews the worker about the work content and the state of the equipment according to the investigation item INV1. The investigator 130 inputs text with the tablet 103. As a result, the tablet 103 generates a memo mx1, which is text data, as the investigation result of the investigation item INV1.

[0025] Next, the investigator 130 conducts an investigation on the investigation item INV2 at the position P2 at the date and time tx2. For example, the investigator 130 inspects the equipment located at the position P2 according to the investigation item INV2, checks whether the work by the worker is appropriate, or conducts an interview with the worker about the work content and the state of the equipment. The investigator 130 images the subject (equipment or worker) with the camera of the tablet 103. Thereby, the tablet 103 generates imaging data pcx2 (still image or moving image) as the investigation result of the investigation item INV2.

[0026] Next, the investigator 130 conducts an investigation on the investigation item INV3 at the position P3 at the date and time tx3. For example, the investigator 130 inspects the equipment located at the position P3 according to the investigation item INV3, checks whether the work by the worker is appropriate, or conducts an interview with the worker about the work content and the state of the equipment. The investigator 130 records the conversation with the worker (which may be a conversation among multiple workers) with the microphone of the tablet 103. Thereby, the tablet 103 generates voice data vx3 as the investigation result of the investigation item INV3.

[0027] Next, the investigator 130 conducts an investigation on the investigation item INV4 at the position P4 at the date and time tx4. For example, the investigator 130 inspects the equipment located at the position P4 according to the investigation item INV4, checks whether the work by the worker is appropriate, or conducts an interview with the worker about the work content and the state of the equipment. The investigator 130 inputs text with the tablet 103 or images the subject with the camera of the tablet 103. Thereby, the tablet 103 generates a memo mx4 and imaging data pcx4 (still image or moving image) as the investigation result of the investigation item INV4.

[0028] Next, the investigator 130 conducts an investigation on the investigation item INV5 at the position P5 at the date and time tx5. For example, the investigator 130 inspects the equipment at the position P5 according to the investigation item INV5, checks whether the work by the worker is appropriate, or interviews the worker about the work content and the state of the equipment. The investigator 130 takes an image of the subject with the camera of the tablet 103 or records the conversation with the worker (which may be a conversation among multiple workers) with the microphone of the tablet 103. As a result, the tablet 103 generates imaging data pcx5 (a still image or a moving image) and audio data vx5 as the investigation results of the investigation item INV5.

[0029] Finally, the investigator 130 ends the on-site investigation at the position P5 at the date and time tx6. The omnidirectional camera 102 stops imaging and recording at the date and time tx6 of the start of the investigation by the operation of the investigator 130.

[0030] Next, the extraction of the omnidirectional imaging data by the server 101 will be described. It is assumed that the server 101 has already acquired the data from the omnidirectional camera 102 and the tablet 103.

[0031] The server 101 extracts the omnidirectional imaging data SI(tx1) within a predetermined time width including the investigation date and time tx1 from the omnidirectional imaging data SIx during the entire investigation time from the investigation date and time tx1 to tx6, based on the investigation date and time tx1 of the investigation item INV1. Then, the server 101 separates the omnidirectional imaging data SI(tx1) into the front imaging data SIf(tx1) and the back imaging data SIb(tx1).

[0032] Server 101 similarly processes the survey dates and times tx1 to tx5 of survey items INV2 to INV5 to separate the omnidirectional imaging data SI(tx2) into front imaging data SIf(tx2) and back imaging data SIb(tx2), separate the omnidirectional imaging data SI(tx3) into front imaging data SIf(tx3) and back imaging data SIb(tx3), separate the omnidirectional imaging data SI(tx4) into front imaging data SIf(tx4) and back imaging data SIb(tx4), and separate the omnidirectional imaging data SI(tx5) into front imaging data SIf(tx5) and back imaging data SIb(tx5).

[0033] <Example of information identification> FIG. 5 is an explanatory diagram showing Example 1 of information identification by server 101. It is assumed that server 101 has already acquired data from omnidirectional camera 102 and tablet 103. FIG. 5 will be described by taking the omnidirectional imaging data SI(tj) cut out at the survey date and time tj as an example.

[0034] Server 101 separates the omnidirectional imaging data SI(tj) into front imaging data SIf(tj) and back imaging data SIb(tj). The front imaging data SIf(tj) includes the facility 501 and the worker W as images. The back imaging data SIb(tj) includes the facility 502 and the workers W2 and W3 as images.

[0035] Server 101 matches the edge data (or 3D model) of facility 501 with the shape information 303 of the site map DB110 by extracting the edge data of facility 501 or performing 3D modeling on facility 501 for the front imaging data SIf(tj). Then, server 101 determines the value Pj of the position information 304 corresponding to the matching shape information 303 as the survey position at the survey date and time tj. Note that "matching" includes cases where the edge data (or 3D model) of facility 501 matches the shape information 303 of the site map DB110, or where the edge data (or 3D model) of facility 501 matches a part of the shape information 303 of the site map DB110.

[0036] Also, when the facility 501 is provided with a two-dimensional code related to the object ID 301, the tablet 103 reads the two-dimensional code and transmits the generated value together with the front imaging data SIf(tj) to the server 101. The server 101 compares the value generated by reading the two-dimensional code with the object ID 301, and if there is a matching object ID 301, it may determine the value Pj of the position information 304 at the survey position at the survey date and time tj.

[0037] Note that since there may be a plurality of the same type of facilities 501 arranged, the server 101 similarly extracts the edge data of the facility 502 by image processing or performs 3D modeling of the facility 502 on the back imaging data SIb(tj), and matches the edge data (or 3D model) of the facility 502 with the shape information 303 of the site map DB110.

[0038] Then, the server 101 uses the position Pj and the value Pk of the position information 304 corresponding to the most similar shape information 303 to identify the position Pj corresponding to the position of the investigator 130 among the positions of the plurality of facilities 501. For example, the server 101 calculates the distance between the position Pk and each position of the several facilities 501, and determines the position Pj of the facility 501 with the shortest distance as the position of the investigator 130. Also, the server 101 searches the site map DB110 for the object name 302 of the facility 501 at the position Pj and holds the object name 302 of the facility 501 as the survey result.

[0039] Also, the server 101 searches the site map DB110 for the object name 302 corresponding to the position Pk of the facility 502 and holds the object name 302 of the facility 502 as out-of-field-of-view information.

[0040] In addition, the server 101 includes the voice data input from the microphone on the front imaging unit 102A side and recorded within a predetermined period including the investigation date and time tj in the investigation result. Further, the server 101 identifies the operator W1 in the front imaging data SIf(tj) as a person by face recognition, presence or absence of movement, or skeleton detection, and may include a character string indicating that the operator W1 is a person (which may be the face image of the operator W1) in the investigation result.

[0041] Similarly for the rear imaging data SIb(tj), the server 101 identifies the operators W2 and W3 in the rear imaging data SIb(tj) as persons by face recognition, presence or absence of movement, or skeleton detection, and holds a character string indicating that the operators W2 and W3 are persons (which may be the face images of the operators W2 and W3) as off-angle-of-view information. In addition, the server 101 may include the voice data input from the microphone on the rear imaging unit 102B side and recorded within a predetermined period including the investigation date and time tj in the off-angle-of-view information.

[0042] FIG. 6 is an explanatory diagram showing an example of information identification by the server 101. Different from FIG. 5, there is no object serving as a clue for identifying the investigation position in the front imaging data SIf(tj). In such a case, the server 101 extracts the edge data of the facility 502 or performs 3D modeling of the facility 502 on the rear imaging data SIb(tj), and matches the edge data (or 3D model) of the facility 502 with the shape information 303 of the site map DB110. Then, the server 101 determines the value Pk of the position information 304 corresponding to the most similar shape information 303 as the investigation position at the investigation date and time tj.

[0043] In addition, the server 101 searches the site map DB110 for the object name 302 corresponding to the position Pk of the facility 502 and holds the object name 302 of the facility 502 as off-angle-of-view information.

[0044] In addition, the server 101 identifies the worker W1 in the front camera data SIf(tj) as a person by face recognition, presence or absence of movement, or skeleton detection, and includes a character string indicating that the worker W1 is a person (which may be the face image of the worker W1) in the investigation result. Further, the server 101 may include, in the investigation result, voice data input from the microphone on the front imaging unit 102A side and recorded within a predetermined period including the investigation date and time tj.

[0045] <Relevance of investigation results on different investigation dates> FIG. 7 is an explanatory diagram showing the relevance of investigation results on different investigation dates. The upper time chart is a time chart showing an example of the actions of the investigator 130 on the investigation date X shown in FIG. 4. The lower time chart is a time chart showing an example of the actions of the investigator 130 on the investigation date Y (≠X) at the same site.

[0046] Similar to FIG. 4, since the upper time chart shows the actions on the investigation date X, it is described using symbols including the lowercase letter x for the investigation date X. The lower time chart shows the actions on the investigation date Y, so symbols including the lowercase letter y for the investigation date Y are used. On the investigation dates X and Y, the investigator 130 generates investigation results using the tablet 103 with the same or different user interfaces for each of the investigation items INV1 to INV5. However, since they are associated with the investigation items INV1 to INV5, the server 101 can obtain them as investigation results of the same investigation item even if the investigation dates are different. Therefore, the user of the server 101 can refer to the investigation results of the same investigation item with different investigation dates.

[0047] <Investigation log DB120> FIG. 8 is an explanatory diagram showing an example of the investigation log DB120. In FIG. 8, as an example, the investigation logs on the investigation dates X and Y shown in FIG. 7 are shown. The investigation log DB120 includes, as fields, the investigation date and time, the investigation position 802, the investigation item 803, the investigation result 804, and the out-of-field-of-view information 805. The combination of the values of each field 801 to 805 in the same row defines one investigation log.

[0048] The investigation date and time 801 is the date and time when the investigation was conducted. The investigation location 802 is the location of the site where the investigation was conducted at the investigation date and time 801. The investigation items 803 (INV1 to INV5) are the contents to be investigated, including, for example, equipment inspections and inspections of work performed by workers.

[0049] The investigation result 804 is data generated by the tablet 103 by the operation of the investigator 130 according to the investigation item 803, and includes a memo 841, imaging data 842, and audio data 843. The off-field-of-view information 805 includes the rear imaging data 851 (SIb(tj)) and the recognition result 852 based on the rear imaging data 851. The server 101 may store the rear imaging data SIb(tj) as it is, or may store the rear imaging data SIb(tj) after removing the overlapping range with the front imaging data.

[0050] The recognition result 852 includes the position Pk of the equipment 502 obtained by image processing of the rear imaging data SIb(tj) and audio data input from the microphone on the rear imaging unit 102B side and recorded.

[0051] FIG. 9 is a flowchart showing an example of an investigation support processing procedure by the server 101. The server 101 acquires data including the investigation date and time 801 and the investigation result 804 from the tablet 103 (step S901) and stores it in the investigation log DB 120 (step S902).

[0052] Next, the server 101 acquires all-spherical imaging data during the investigation period from the all-spherical camera 102 (step S903). The server 101 cuts out all-spherical imaging data within a predetermined time width including the investigation date and time 801 in units of the investigation date and time 801 from the all-spherical imaging data during the investigation period, and separates the cut-out all-spherical imaging data into front imaging data and rear imaging data (step S904).

[0053] Next, if an object exists in the front imaging data, the server 101 identifies the investigation position at the investigation date and time 801 using at least the front imaging data among the front imaging data and the rear imaging data. If no object exists in the front imaging data, the server 101 identifies the investigation position at the investigation date and time 801 using the rear imaging data (step S905).

[0054] Next, the server 101 performs image recognition using the rear imaging data 851 (step S906). Then, the server 101 stores the recognition result 852 obtained by the image recognition (step S906) together with the rear imaging data 851 in the investigation log DB 120 in association with the investigation date and time 801 (step S907).

[0055] Note that the tablet 103 can display the investigation position 802, the investigation item 803, the investigation result 804, and the off - field - of - view information 805 on a separate screen when playing back the omnidirectional imaging data SIx for the entire investigation time at the investigation dates and times tx1 to tx6 by accessing the investigation log DB 120.

[0056] FIG. 10 is an explanatory diagram showing a display example on the tablet 103. FIG. 10 shows a playback example of the omnidirectional imaging data SIx for the entire investigation time at the investigation dates and times tx1 to tx6 on the investigation day X. The tablet 103 displays a playback area 1001 and a playback bar 1002 on the display screen 1000. A playback button 1011 and a stop button 1012 are displayed in the playback area 1001. By pressing the playback button 1011, the omnidirectional imaging data SIx is played back, and by pressing the stop button 1012, the playback is stopped.

[0057] The playback bar 1002 is a time axis indicating the playback time length of the omnidirectional imaging data SIx. On the playback bar 1002, a position marker 1020 moves on the playback bar 1002 according to the playback position of the omnidirectional imaging data SIx. Investigation result icons 1021 to 1027 are displayed on the playback bar 1002.

[0058] FIG. 11 is an explanatory diagram showing an example of displaying the survey result when the survey result icon 1021 is pressed. When the survey result icon 1021 is pressed, a memo screen 1100 showing a memo mx1 which is the survey result 804 of the survey item INV1 is displayed.

[0059] Although not shown in the figure, similarly, when the survey result icon 1022 is pressed, imaging data pcx2 which is the survey result 804 of the survey item INV2 is displayed on a separate screen. When the survey result icon 1023 is pressed, audio data vx3 which is the survey result 804 of the survey item INV3 is played. When the survey result icon 1024 is pressed, a memo mx4 which is the survey result 804 of the survey item INV4 is displayed on a separate screen. When the survey result icon 1025 is pressed, imaging data pcx4 which is the survey result 804 of the survey item INV4 is displayed on a separate screen. When the survey result icon 1026 is pressed, imaging data pcx5 which is the survey result 804 of the survey item INV5 is displayed on a separate screen. When the survey result icon 1027 is pressed, audio data vx5 which is the survey result 804 of the survey item INV5 is played.

[0060] Also, the investigator 130 can rotate the omnidirectional imaging data SIx by swiping the playback area 1001 to the left and right. Thereby, when the playback position is the survey date and time 801, the tablet 103 can simultaneously display the survey result 804 of the survey date and time 801 and the back imaging data 851 of the same survey date and time 801. Also, at this time, the tablet 103 may display the corresponding recognition result 852.

[0061] In this way, according to the present embodiment, the survey result 804 at the same survey date and time 801 within the viewing angle of the investigator 130 and the off-view angle information 805 can be associated. Therefore, it is possible to improve the accuracy of the on-site actual situation survey. For example, after the survey, the investigator 130 can discover omissions in the survey or identify key points during re-survey. Also, it is possible to eliminate the need for additional surveys by discovering omissions in the survey.

[0062] Note that the present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the configurations described. Also, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Further, the configuration of another embodiment may be added to the configuration of one embodiment. Also, for part of the configuration of each embodiment, addition, deletion, or replacement with other configurations may be made.

[0063] In addition, each of the above-described configurations, functions, processing units, processing means, etc. may be realized in hardware, for example, by designing part or all of them with an integrated circuit, or may be realized in software by a processor interpreting and executing a program for realizing each function.

[0064] Information such as programs, tables, files, etc. for realizing each function can be stored in a storage device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0065] Also, control lines and information lines show those considered necessary for explanation, and do not necessarily show all control lines and information lines necessary for implementation. In reality, it may be considered that almost all configurations are interconnected.

Explanation of Reference Numerals

[0066] 100 Investigation Support System 101 Server 102 Omnidirectional Camera 103 Tablet 130 Investigator 200 Investigation Support Device 201 Processor 202 Storage Device 302 Object Name 303 Shape Information 304 Location Information 801 Investigation Date and Time 802 Investigation Location 803 Investigation Items 804 Investigation Results 805 Information outside the Field of View 841 Memo 842 Imaging Data 843 Audio Data 851 Rear Imaging Data 852 Recognition Results 852 Information outside the Field of View

Claims

1. An investigation support device having a processor that executes a program and a storage device that stores the program, is accessible to a database that stores feature information and position information of an object, wherein the processor performs a first acquisition process of acquiring an investigation result including at least one of text data, imaging data, and audio data in an investigation at an investigation date and time, a second acquisition process of acquiring first imaging data captured from the front of an investigator at the investigation date and time and second imaging data captured from the back of the investigator at the investigation date and time, a specifying process of specifying an investigation position at the investigation date and time based on at least one of the first imaging data and the second imaging data acquired by the second acquisition process, an association process of associating the investigation result acquired by the first acquisition process, the second imaging data, and the investigation position specified by the specifying process with the investigation date and time, and executes, in the specifying process, the processor if there is a second object having second feature information that matches the feature information of the first object in the database in the first imaging data, determines the position information of the first object as the investigation position at the investigation date and time, if there are a plurality of second objects having second feature information that matches the first feature information of the first object in the database in the first imaging data, and there is a fourth object having fourth feature information that matches the third feature information of the third object in the database in the second imaging data, based on the position information of the third object, determines the position information of a specific second object among the plurality of second objects as the investigation position at the investigation date and time, characterized in that it is an investigation support device.

2. The investigation support device according to claim 1, wherein in the specifying process, the processor determines the position information of the third object as the investigation position at the investigation date and time if there is no second object having second feature information that matches the first feature information of the first object in the database in the first imaging data, and there is a fourth object having fourth feature information that matches the third feature information of the third object in the database in the second imaging data, characterized in that it is an investigation support device.

3. The investigation support device according to claim 1, wherein the processor performs a reproduction process of reproducing combined imaging data obtained by combining the first imaging data and the second imaging data, Display processing for displaying combined imaging data reproduced by the reproduction processing, a time axis indicating the reproduction time length by the reproduction processing, and an icon related to the investigation result arranged at a reproduction position corresponding to the investigation date and time on the time axis; An investigation support device characterized by executing the above.

4. The investigation support device according to claim 3, In the display processing, when the icon displayed by the display processing is selected, the processor displays the investigation result. An investigation support device characterized by the above.

5. An investigation support method by an investigation support device having a processor that executes a program and a storage device that stores the program, The investigation support device is accessible to a database that stores feature information and position information of an object, The processor, A first acquisition process for acquiring an investigation result including at least one of text data, imaging data, and audio data in an investigation at the investigation date and time; A second acquisition process for acquiring first imaging data captured from the front of the investigator at the investigation date and time and second imaging data captured from the back of the investigator at the investigation date and time; A specifying process for specifying an investigation position at the investigation date and time based on at least one of the first imaging data and the second imaging data acquired by the second acquisition process; An association process for associating the investigation result acquired by the first acquisition process, the second imaging data, and the investigation position specified by the specifying process with the investigation date and time; Execute, In the specifying process, the processor, When there is a second object with second feature information that matches the feature information of the first object in the database in the first imaging data, the position information of the first object is determined as the investigation position at the investigation date and time; When there are a plurality of second objects with second feature information that matches the first feature information of the first object in the database in the first imaging data, and there is a fourth object with fourth feature information that matches the third feature information of the third object in the database in the second imaging data, based on the position information of the third object, the position information of a specific second object among the plurality of second objects is determined as the investigation position at the investigation date and time. An investigation support method characterized by the above.

Images