Information processing methods, information processing devices, information processing systems, and computer programs

Abstract

This invention provides an information processing method, an information processing device, an information processing system, and a computer program that improve the accuracy of recognizing hazards in electrical equipment construction. [Solution] The information processing method includes acquiring on-site images of electrical equipment construction, using a learning model that has been trained to output recognition results of hazardous areas shown in the images when an image is input, outputting the recognition results for the acquired on-site images, acquiring corrected data for the recognition results, and retraining the learning model based on the corrected data and a plurality of training images generated from the acquired on-site images.

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing apparatus, an information processing system, and a computer program for improving the recognition accuracy of risk factors in electrical equipment work.

Background Art

[0002] The site of electrical equipment work is often at a high place such as the second floor between houses where service wires are connected, the roof, or the tip of a utility pole. Work related to outdoor electrical equipment involves various risks such as falling or stepping off.

[0003] Patent Document 1 discloses a system that provides an appropriate work plan to support safety management because there is a risk of insufficient attention to safety management due to being rushed for work.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Take photos at each stage of the investigation stage, immediately before the start of work, and during work, and proceed with the work while performing risk detection based on the recognition of the objects shown in the photos to recognize risk factors and proceed with the work safely. In this case, the brightness of the work site related to outdoor electrical equipment changes depending on morning, noon, evening, and weather differences, and it is not easy to recognize risk factors from the photos.

[0006] An object of the present disclosure is to provide an information processing method, an information processing apparatus, an information processing system, and a computer program for improving the recognition accuracy of risk factors in electrical equipment work.

Means for Solving the Problems

[0007] An information processing method according to one embodiment of the present disclosure includes the following steps: acquiring on-site images of electrical equipment construction work; using a learning model that has been trained to output recognition results of hazardous areas in the images when an image is input; outputting the recognition results for the acquired on-site images; acquiring corrected data for the recognition results; and retraining the learning model based on the corrected data and a plurality of training images generated from the acquired on-site images. [Effects of the Invention]

[0008] This disclosure makes it possible to improve the accuracy of identifying hazards in electrical equipment installation work. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the information processing system according to the first embodiment. [Figure 2] This is a block diagram showing the configuration of an information processing device. [Figure 3] This is an overview diagram of the learning model. [Figure 4] This is a block diagram showing the configuration of a terminal device. [Figure 5] This flowchart shows an example of a processing procedure performed by an information processing system. [Figure 6] This flowchart shows an example of a processing procedure performed by an information processing system. [Figure 7] This flowchart shows an example of a retraining process in an information processing device. [Figure 8] An example of a screen displayed on the terminal device's display unit is shown. [Figure 9] An example of a screen displayed on the terminal device's display unit is shown. [Figure 10] An example of a screen displayed on the terminal device's display unit is shown. [Figure 11] This figure shows an example of the content of training images. [Figure 12] This figure shows an example of the content of training images. [Figure 13] It is a diagram showing an example of the content of a learning image. [Figure 14] It is a diagram showing an example of the content of a learning image. [Figure 15] It is a diagram showing an example of the content of a learning image. [Figure 16] It is a block diagram showing the configuration of the information processing apparatus according to the second embodiment. [Figure 17] It is a schematic diagram of an image generation model. [Figure 18] It is a flowchart showing an example of the relearning process in the second embodiment. [Figure 19] It is a block diagram showing the configuration of the information processing apparatus according to the third embodiment. [Figure 20] It is a flowchart showing an example of the processing procedure by the information processing system according to the third embodiment. [Figure 21] It is a flowchart showing an example of the processing procedure by the information processing system according to the third embodiment. [Figure 22] It is a flowchart showing an example of the relearning process in the information processing apparatus according to the third embodiment. [Figure 23] An example of a screen displayed on the display unit of the terminal device according to the third embodiment is shown. [Figure 24] It is a schematic diagram of the information processing system according to the fourth embodiment. [Figure 25] It is a block diagram showing the configuration of the terminal device according to the fourth embodiment. [Figure 26] It is a flowchart showing an example of the recognition process for a dangerous location according to the fourth embodiment.

Mode for Carrying Out the Invention

[0010] The present disclosure will be specifically described with reference to the drawings showing its embodiments.

[0011] (First Embodiment) Figure 1 is a schematic diagram of the information processing system 100 of the first embodiment. The information processing system 100 is a system that aims to reduce the occurrence of accidents during work to zero by presenting workers with hazardous factors at the work site from recorded data including photographs or videos taken by workers at the electrical equipment construction site.

[0012] The information processing system 100 includes a terminal device 2 used by workers and an information processing device 1 that communicates with the terminal device 2 via a network N. The network N includes a public communication network, a carrier network, and a local network. The network N may also include a dedicated line.

[0013] Terminal device 2 is a device that can connect to or has a camera 26 built into it. Camera 26 is used to photograph the site and surroundings of the electrical equipment installation work performed by the workers. Terminal device 2 acquires image data of the photographs taken from camera 26. Terminal device 2 is equipped with a communication unit for communication via network N and transmits the photographs taken at the site to information processing device 1. Information processing device 1 uses a learning model M1 for image recognition to recognize objects related to electrical work that are captured in the photographs. The target of image recognition is not limited to still images such as photographs, but may also be videos. Terminal device 2 is equipped with a display unit 23, which acquires the object recognition results from information processing device 1 and displays the recognition results on the display unit 23 so that they can be seen by the workers. Terminal device 2 may further be equipped with a function to support the creation of a pre-construction plan or the creation of pre- and post-construction reports using the image data of photographs or videos.

[0014] In the information processing system 100 disclosed herein, the terminal device 2 accepts corrections to the recognition results from the information processing device 1 based on the worker's on-site recognition, thereby improving the recognition accuracy of the learning model M1 in the information processing device 1.

[0015] The following describes the configuration and processing details of the information processing device 1 for improving the recognition accuracy of image data using the learning model M1.

[0016] Figure 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 comprises a processing unit 10, a storage unit 11, and a communication unit 12. The information processing device 1 uses a server computer. In the following description, the information processing device 1 is described as being composed of a single server computer, but it may also be configured such that multiple server computers are connected via a local network LN and perform distributed processing to function as a single information processing device 1. The information processing device 1 functions as an on-premise server, but it may also function as a cloud server while ensuring security.

[0017] The processing unit 10 includes one or more processors such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit). The processing unit 10 also includes memory, which is a temporary storage medium such as SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory). The processing unit 10 may be configured as a single hardware (SoC: System On a Chip) integrating the processor, memory, storage unit 11, and communication unit 12.

[0018] The storage unit 11 is a relatively large-capacity non-temporary storage medium, such as a hard disk or SSD (Solid State Drive). The storage unit 11 stores the program (program product) necessary for the processing unit 10 to execute processing, and reference setting data. The program product includes an information processing program P1 and a learning model M1 for image recognition.

[0019] The information processing program P1 included in the program product is a program that, when read into memory and executed by the processing unit 10, causes a general-purpose computer to function as the information processing device 1 of this disclosure, which performs various processes described later.

[0020] The learning model M1 is a model that has been trained by the information processing device 1 or another device to output the recognition result of hazardous areas related to electrical equipment construction depicted in an image when an image is input. The input image may be not only a still image, but also a video that includes images before and after in chronological order.

[0021] The information processing program P1 and learning model M1 stored in the memory unit 11 may be obtained by the processing unit 10 reading the information processing program P9 and learning model M9 stored in the computer-readable non-temporary storage medium 9 and storing them in the memory unit 11. Alternatively, the information processing program P1 and learning model M1 may be obtained by the processing unit 10 downloading them from a download server via the communication unit 12 and storing them in the memory unit 11.

[0022] The communication unit 12 enables communication via the network N. The communication unit 12 may be a wired network card or a wireless communication device for Wi-Fi. The communication unit 12 may also be a wireless communication device connected to a carrier network. The processing unit 10 can send and receive data with the terminal device 2 via the communication unit 12.

[0023] Figure 3 is an overview diagram of the learning model M1. The learning model M1 is a neural network model that takes image data as input and outputs data indicating the extent of the dangerous area as a result of dangerous area recognition. As described above, the image data may be not only still image data but also video data that includes images before and after in chronological order. The learning model M1 may output the data indicating the extent of the dangerous area as coordinate data within the image, or it may output image data in which the extent of the dangerous area is represented by specific pixel values. If no dangerous area is recognized, the learning model M1 outputs data indicating a range of zero or image data with pixel values ​​different from the specific pixel values. The learning model M1 includes an input layer, an output layer, and intermediate layers including convolutional layers for the input image between them.

[0024] The learning model M1 is trained using training data that includes images of electrical equipment installation sites collected in advance, and data specifying the areas to be recognized as hazardous in each image. The learning model M1 is trained by updating the parameters of the intermediate layer by backpropagating the error between the data indicating the range of hazardous areas in the image, which is output when training data images are given, and the data indicating the pre-specified range for the actual image. The learning model M1 may also output the result of determining the type of hazardous area, not just the range of the hazardous area. If the learning model M1 outputs image data in which the range of the hazardous area is represented by specific pixel values, it may also output a segmented image in which the pixel values ​​are distinguished by the type of hazardous area.

[0025] Here, a hazardous location refers to a place where there is an object that may cause a hazard (hazard factor) such as a work-related accident or injury, including falls and drops. The type of hazardous location may be the result of object identification. The object may be a wall, column, groove, hole (space), or any object that a worker recognizes as a hazardous location. Hazardous locations are not limited to fixed objects such as walls and columns shown in the image, but may also be places where there is equipment such as heavy machinery, or objects carried by workers such as vehicles or tools that may be hazard factors in electrical equipment installation work. The type of hazardous location may also be the type of accident (type of hazard) that occurs as a result of that object as a hazard factor. In this case, examples of accident types include "falls," "tipping," "collisions," "falling objects," "collapse," "being hit," "being caught in," "cuts," "abrasions," "stepping through," "drowning," "contact with high-temperature or low-temperature objects," "contact with hazardous substances," "electric shock," "explosion," "rupture," "fire," "traffic accident (road)," "traffic accident (other)," "recoil from movement / unnatural movement," etc.

[0026] The processing unit 10 can use the learning model M1 to recognize the presence or absence of hazardous areas and, if present, their extent, by providing the learning model M1 with on-site images taken at the site. The processing unit 10 can utilize the results when the learning model M1 outputs a result indicating the type of hazardous area.

[0027] The M1 learning model is retrained to improve its recognition accuracy using newly captured on-site images (still images or videos, etc.). Details of the retraining process will be described later.

[0028] Figure 4 is a block diagram showing the configuration of terminal device 2. Terminal device 2 is a smartphone, tablet, or personal computer. Preferably, terminal device 2 is a portable device with a built-in camera that is brought to the electrical equipment installation site, but it may also be a desktop personal computer that captures images from a camera that is carried separately.

[0029] The terminal device 2 comprises a processing unit 20, a storage unit 21, a communication unit 22, a display unit 23, an operation unit 24, an audio input / output unit 25, and a camera 26.

[0030] The processing unit 20 includes one or more processors such as CPUs, MPUs, and GPUs. The processing unit 20 also includes memory, which is a temporary storage medium such as SRAM or DRAM. The processing unit 20 may be configured as a single hardware (SoC: System On a Chip) integrating the processor, memory, storage unit 21, and communication unit 22.

[0031] The storage unit 21 is a relatively large-capacity non-volatile storage area such as an SSD or flash memory. The storage unit 21 stores the program (program product) necessary for the processing unit 20 to execute processing, and reference setting data. The program product includes the terminal program P2. The terminal program P2 may also include a web client program. The terminal program P2 may be downloaded by the processing unit 20 from the information processing device 1 or download server via the communication unit 22 and stored in the storage unit 21, or it may be a terminal program P8 stored on the storage medium 8 that the processing unit 20 reads and stores in the storage unit 21.

[0032] The communication unit 22 enables communication with the information processing device 1 via the network N. The communication unit 22 may be a wired network card or a WiFi wireless communication device. The communication unit 22 may also be a wireless communication device connected to a carrier network.

[0033] The display unit 23 is a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 23 may also be a touch panel-integrated display. The processing unit 20 displays an operation screen based on the terminal program P2 and on-site images captured by the camera 26 on the display unit 23.

[0034] The operation unit 24 is a user interface that can input and output to the processing unit 20. The operation unit 24 is, for example, a touch panel built into the display unit 23. The operation unit 24 may include a pointing device such as a mouse and a keyboard, or it may also include physical buttons, switches, and physical dials. The operation unit 24 may be configured to accept voice operation using the microphone and voice recognition processing unit of the voice input / output unit 25.

[0035] The audio input / output unit 25 includes a speaker and a microphone. The processing unit 20 or the audio input / output unit 25 itself may have a speech recognition function that converts audio collected using the microphone into text data. The processing unit 20 can output sound effects and voice using the speaker.

[0036] Camera 26 outputs image data using an image sensor that corresponds to visible light and / or near-infrared light. Camera 26 may also output video image data. Camera 26 may output image data associated with time information measured by a built-in timer, or store it in a built-in storage medium. Camera 26 is a camera that is detachable from the terminal device 2, and when connected to the terminal device 2, it may output the image data stored in the camera 26's built-in storage medium to the terminal device 2.

[0037] The following describes the processes performed by the information processing system 100 configured in this way. The worker brings the terminal device 2 and goes to the site in advance of the work to conduct a safety pre-assessment, to determine the necessary equipment, tools, heavy machinery to be used, and construction methods, or to actually carry out the work. The worker takes photographs or videos of the site using the terminal device 2 or the camera 26 alone. When the worker starts the terminal program P2 on the terminal device 2 and performs the operation to import the captured site images, the information processing system 100 executes the following processes. In the following description, "site images" are not limited to still images such as photographs, but also include videos. Figures 5 and 6 are flowcharts showing an example of the processing procedure by the information processing system 100.

[0038] The processing unit 20 acquires image data of the electrical equipment construction site captured by the camera 26 (step S201). In step S201, the processing unit 20 may acquire time information of the date and time of capture as metadata for the image data. The processing unit 20 transmits the acquired image data along with a recognition request to the information processing device 1 (step S202).

[0039] The processing unit 10 of the information processing device 1 performs the following processing based on the information processing program P1. The processing unit 10 receives image data of the field image along with a recognition request (step S101). The processing unit 10 provides the received image data to the learning model M1 (step S102) and obtains data indicating the extent of the hazardous area output from the learning model M1 (step S103). If the identification result of the object related to the hazardous area or the type of accident corresponding to the hazardous factor is also output as the type of hazardous area, the processing unit 10 also obtains this data in step S103.

[0040] The processing unit 10 creates an image by superimposing an object representing the area of ​​the hazardous area onto the on-site image, based on the image data acquired in step S101 and the data indicating the area of ​​the hazardous area acquired in step S103 (step S104). The processing unit 10 transmits the image data of the on-site image with the area of ​​the hazardous area superimposed to the terminal device 2 that made the recognition request in step S101 (step S105).

[0041] The processing unit 20 of the terminal device 2 receives image data (step S203) and, based on the received image data, displays a screen on the display unit 23 showing information about hazardous areas in the captured on-site image (step S204). The screen displayed in step S204 includes an on-site image with the area of ​​the hazardous area superimposed.

[0042] The processing unit 20 displays an interface that accepts corrections to the recognition results of hazardous areas (step S205). In step S205, the processing unit 20 outputs an interface (layer) on the screen that accepts operations to change the range, delete the range, or add the range on the site image overlaid with the ranges of the hazardous areas. The processing unit 20 accepts input of text from the worker describing the hazardous factors at the hazardous area recognized (step S206). The processing unit 20 may accept the selection of the type of hazardous area instead of inputting text describing the hazardous area. As described above, the selection of the type of hazardous area involves the worker selecting the object identification result from object names such as walls, columns, ditches, holes (spaces), etc., or entering it as text, in order to learn the identification of the object itself. The selection of the type of hazardous area may also be selected from the types of accidents that occur with that object as a hazardous factor. If there are no corrections, the processing in step S206 is omitted.

[0043] The processing unit 20 determines whether or not the recognition result has been corrected (step S207). If it is determined that no correction has been made (S207: NO), the processing unit 20 stores in the storage unit 21 the data of the on-site image with the area of ​​the hazardous area displayed in step S204 superimposed, the data of the on-site image before superimposition in step S201, and the data of the shooting environment including information on the electrical equipment to be constructed and information on the date and time of shooting (step S208). The processing unit 20 terminates the process.

[0044] This allows workers to view on-site images with the hazardous areas superimposed on them using the terminal device 2, and to use the image data stored in the storage unit 21 when creating records before or during construction.

[0045] If it is determined in step S207 that a correction has been made (S207:YES), the processing unit 20 stores the correction data, the on-site image data acquired in step S201, and the shooting environment data, including information on the electrical equipment to be worked on and the date and time of shooting, in the storage unit 21 (step S209). The correction data is data indicating the extent of the hazardous area and / or text describing the hazardous area. The storage of data in step S209 may be skipped. If the learning model M1 is trained to output the recognition result of the hazardous area while determining the type of hazardous area, the correction data includes the type of hazardous area selected (type of object or type of accident).

[0046] Even if the recognition results from the learning model M1 are modified, workers can still view the on-site image with the hazardous area overlaid on the terminal device 2, and use the modified image data stored in the storage unit 21 when creating records before, during, or after construction.

[0047] The processing unit 20 transmits the corrected data to the information processing device 1 (step S210). In step S210, the processing unit 20 may transmit the field image again, or it may transmit identification data indicating that it is the same data as the field image transmitted in step S202.

[0048] The processing unit 10 of the information processing device 1 receives the correction data (step S106). The received correction data and the on-site image received in step S101 are stored in the storage unit 11 as training data (step S107).

[0049] The processing unit 10 performs a retraining process using training data that includes the corrected data and the field image (step S108), and then terminates the process.

[0050] Figure 7 is a flowchart showing an example of the retraining process in the information processing device 1. The processing procedure shown in Figure 7 corresponds to the details of step S108 of the processing procedure shown in Figures 5 and 6.

[0051] The processing unit 10 acquires a set of training data consisting of a field image and corrected data (step S801). The processing unit 10 generates a training image by rotating the field image of the training data (step S802). In step S802, the processing unit 10 rotates the field image so that straight lines along the boundaries of the road surface, floor surface, and ground in the field image align with the vertical or horizontal direction in the image (see Figure 11). If the field image contains an object extending in a substantially vertical direction, the processing unit 10 rotates the field image so that the outline of that object aligns with the vertical or horizontal direction in the image.

[0052] The processing unit 10 generates training images by changing the brightness of the field images of the training data (step S803). In step S803, the processing unit 10 performs this action to simulate and easily introduce the effects of different seasons, times of day, or weather conditions at the time of shooting into the field images. Therefore, if the original field image was taken in the morning, the processing unit 10 may darken the brightness to make it appear as if it was taken in the evening, and if the original field image was taken in the evening, it may brighten the brightness.

[0053] The processing unit 10 generates training images by overlaying semi-transparent, solid-color rectangles onto the field images of the training data, varying the color and / or transparency (step S804). The training images with the solid-color rectangles are generated with the intention of reproducing changes in the shooting environment, such as season, time of day (duration), and weather, through processing such as optical filters and digital filters (see Figure 13).

[0054] The processing unit 10 generates a training image by enlarging a portion of the field image from the training data (step S805).

[0055] The processing unit 10 stores each of the training images generated in steps S802-S805 as new training data by combining it with the corrected data acquired in step S801 (step S806). As described above, the corrected data may include not only the range, but also text (labels) that describe the object to be identified, and data that identifies the type of accident.

[0056] The processing unit 10 updates the parameters of the learning model M1 using the training data stored in step S806 (step S807), and then terminates the retraining process. The processing unit 10 then returns to the processes shown in Figures 5 and 6 and terminates.

[0057] Of the processing steps shown in Figure 7, the training image generation process in steps S802-S805 does not necessarily have to be performed on a single field image of the training data. The processing unit 10 may generate all patterns of training images, such as training images with only steps S802-S805 performed, six patterns of training images with two of steps S802-S805 performed, four patterns of training images with three of steps S802-S805 performed, or one training image with all of steps S802-S805 performed, or it may perform them randomly.

[0058] The processing unit 10 may generate all patterns while the learning model M1 has low recognition accuracy for the range of dangerous areas, i.e., while the learning progress is shallow. When learning progresses and the recognition accuracy reaches a value of 1 or higher, it may generate about half of the patterns. When it is judged that learning has progressed sufficiently, if the recognition accuracy reaches a value of 2 or higher, which is higher than the first value, it may be controlled not to perform retraining.

[0059] The processing unit 10 of the information processing device 1 may accept a setting to determine which of the methods in steps S802-S805 will be used to generate the training images, that is, a setting for the type of training images to be generated. The processing unit 10 may also accept a setting for how many training images to generate. When accepting a setting, the processing unit 10 may output the progress of the training of the training model M1 and output a setting screen so that the user can check the progress while making the setting.

[0060] The processing unit 10 of the information processing device may, at predetermined intervals, i.e., periodically, use the data of the field images transmitted from the terminal device 2 and the data on dangerous areas drawn on the field images by the workers as training data to perform the processing shown in Figure 7. The processing shown in Figure 7 may also be performed each time the amount of unused field images transmitted from the terminal device 2 for retraining reaches a predetermined amount.

[0061] By retraining using the processing procedure shown in Figure 7, it is possible to improve the recognition accuracy of the learning model M1, which recognizes objects corresponding to dangerous areas perceived by on-site workers from images.

[0062] The processing procedure shown in Figures 5-7 will be explained with specific examples. Figures 8-10 show examples of screens displayed on the display unit 23 of the terminal device 2. Figures 8-10 show an example of an operation screen 230 displayed when the terminal program P2 is launched on the terminal device 2. The operation screen 230 shown in Figure 8 shows the field image I1 captured by the camera 26 as a line diagram. The field image I1 includes the road, the roadside ditch, and the utility pole standing on the road in its shooting range.

[0063] The operation screen 230 shown in Figure 8 includes a hazard recognition button 231 for sending an image to the information processing device 1 to perform a process that recognizes the area indicating a hazard. When the hazard recognition button 231 is selected by the operation unit 24, the processing unit 20 sends the image data of the on-site image I1 to the information processing device 1 (S202).

[0064] The operation screen 230 shown in Figure 9 shows the result of recognition processing performed by the information processing device 1 on the field image I1 shown in Figure 8. In the example in Figure 9, no object representing the extent of the hazardous area is displayed, and the hazardous area is not recognized. The operation screen 230 shown in Figure 9 includes a recognition correction button 232 for outputting an interface that accepts corrections regarding the extent of the hazardous area.

[0065] In the operation screen 230 shown in Figure 10, the recognition correction button 232 is selected in the operation screen 230 shown in Figure 9, and an interface 233 that accepts corrections regarding the range of the hazardous area is displayed. Interface 233 may already be displayed in the state shown in Figure 9, where the recognition results are displayed. In the example in Figure 10, interface 233 is represented by a rectangular object whose position and size can be changed. In the example in Figure 10, the worker has selected, for example, a drainage ditch included in the site image I1 as the range of the hazardous area.

[0066] The operation screen 230 shown in Figure 10 includes an input interface 234 for text describing the hazardous factors at a hazardous location. In Figure 10, the text "Danger of slipping in the gutter" is entered into the input interface 234. The operation screen 230 shown in Figure 10 includes a send button 235 for sending the correction result to the information processing device 1. When the send button 235 is selected by the operation unit 24, the processing unit 20 of the terminal device 2 determines that a correction has been made (S207: YES). The processing unit 20 stores the coordinate data of the rectangular object in the image corresponding to the interface 233 and the text entered in the input interface 234 as correction data in the storage unit 21 (S209) and sends it to the information processing device 1 (S210). The input interface 234 may also contain descriptions of what the worker felt was dangerous, descriptions of accidents that have actually occurred, and methods for dealing with this hazardous location for sharing with others. This allows specific measures taken by workers who perceive a dangerous area as dangerous (such as placing objects like traffic cones to block the drainage ditch, as in the example in Figure 10) to be output and shared with other workers when they encounter a similar environment.

[0067] The operation screen 230 shown in Figure 10 includes an accident type selection interface 238 corresponding to the type of hazardous location. In Figure 10, the selection interface 238, in this case, allows the learning model M1 to be trained using labeled (annotated) training data of the selected type, and to output identification results of accidents that are likely to occur, corresponding to the recognized object as the type of hazardous location.

[0068] Figures 8-10 illustrate an example of generating training images during retraining based on the correction data transmitted. Figures 11-15 show examples of the content of training images. The processing unit 10 of the information processing device 1 generates multiple patterns of training images based on the correction data, which includes the range of the dangerous area shown in Figure 10.

[0069] Figure 11 shows training image I2, which is a rotated version of the on-site image. Figure 11 shows training image I2, which is a rotated version of the on-site image I1 shown in Figure 8, in which a straight line along the gutter is rotated to align with the vertical direction of the image. In images of outdoor electrical equipment construction sites, objects that extend in the depth direction, such as road gutters, appear at an angle within the image, unlike photographs taken from directly above with the gutter itself as the subject, or images with only a utility pole as the subject, resulting in reduced recognition accuracy. This is because elements other than the subject appearing at an angle are included in image I1, which reduces accuracy. For example, gutters can be perceived as dangerous areas by workers because there is a risk of them tripping while walking and looking up at the top of a utility pole. Even in images where the utility pole is included in the shooting range, gutters may be recognized as dangerous areas by workers. For objects in photographs related to outdoor electrical equipment construction, particularly long objects such as gutters extending in the depth direction, the accuracy of object recognition in the on-site image I1 can be improved by training the model with a training image I2 that rotates the straight line along the depth direction of the object to align with the vertical or horizontal direction of the image.

[0070] Figure 12 shows training image I3, in which the brightness of the field image has been altered. Figure 12 shows training image I3, which has been darkened compared to field image I1 shown in Figure 8. As mentioned above, the brightness and saturation of field image I1 change depending on the season, time of day, or weather, even when the same object is photographed at the same angle of view. Therefore, by deliberately generating training image I3 with altered brightness or saturation and using it for retraining, it is possible to improve the recognition accuracy of field image I1 taken under diverse seasonal, time of day, or weather conditions.

[0071] Figure 13 shows training image I4, which is a field image overlaid with a semi-transparent, solid-color rectangle. Training image I4, shown in Figure 13, is created by overlaying a color filter onto field image I1. Training image I4 is created with the intention of reproducing changes in the shooting environment, such as season, time of day (duration), and weather, through processing such as optical filters and digital filters. By retraining using training image I4 shown in Figure 13, the recognition accuracy for field image I1 taken under various seasonal, time, or weather conditions can be improved, similar to changes in brightness or color.

[0072] Figure 14 shows training image I5, which was generated by enlarging a portion of the field image I1. Training image I5 shown in Figure 14 is an image created by cropping and enlarging only the crossarm portion of the utility pole visible in the field image I1. As shown in Figure 14, by retraining using training image I5, which contains only the object to be recognized, in this case the crossarm portion, it becomes possible to adjust the parameters of the learning model M1 to respond to the features of the object to be recognized (in this case the utility pole), ignoring the features of other parts, thereby improving recognition accuracy and speeding up the learning process. In addition, by using only the characteristic crossarm portion, the amount of training data required to achieve the same level of recognition accuracy can also be reduced.

[0073] Figure 15 shows another example of a training image I6 generated by enlarging a portion of the field image I1. The training image I6 shown in Figure 15 is an image obtained by rotating the field image I1 as shown in training image I2 in Figure 11, and then cropping the gutter in multiple places along its length. By retraining using training image I6, which contains only the object to be recognized, in this case the gutter portion, it becomes possible to adjust the parameters of the learning model M1 to respond strongly to the features of the object to be recognized (in this case the gutter), thereby improving recognition accuracy.

[0074] Any of the training images I2 to I6 shown in Figures 11-15 may be generated by the information processing device 1, or by an operator of the information processing device 1.

[0075] In this way, by retraining using corrected data on hazardous areas perceived by workers on-site and various training images I2-I6, the accuracy of recognizing hazardous areas in photographs taken during outdoor electrical equipment installation can be improved. Using the information processing system 100, which can identify hazardous areas using the learning model M1, terminal device 2 can provide each worker with an opportunity to recognize hazards they may not have noticed themselves. In a work environment that is normal for the customer, the information processing device 1 can provide a third-party perspective to warn about other factors that may be overlooked by both the customer and the workers, thereby further strengthening safety. Relying solely on the individual abilities of workers and their efforts to "be careful" is insufficient for improving on-site safety. As part of safety education, it is also possible to learn from examples of hazardous areas previously stored in the information processing device 1 using terminal device 2. In this way, by understanding the characteristics of worker perception and creating displays that are easy to see and strongly appeal to the five senses, individual workers' awareness of hazardous areas can be improved.

[0076] (Second Embodiment) In the second embodiment of the information processing system 100, an image generation model is used to generate images, and training images are amplified to enable accurate recognition even from a small number of field images I1. The configuration of the second embodiment of the information processing system 100 is the same as that of the first embodiment of the information processing system 100, except for a part of the information processing device 1 which will be described later. Therefore, the same reference numerals are used for common components and detailed explanations are omitted.

[0077] Figure 16 is a block diagram showing the configuration of the information processing device 1 in the second embodiment. In the information processing system 100 of the second embodiment, the program product stored in the storage unit 11 of the information processing device 1 includes an image generation model M2. The image generation model M2 is a model that takes image data and text or images indicating settings as input and converts the original image data according to the settings. The image generation model M2 may operate inside the information processing device 1, or part or all of it may operate on an external server. The image generation model M2 may be an image generation model M8 that was stored in a non-temporary storage medium 9, similar to the information processing program P1 and the learning model M1, which the processing unit 10 reads and stores in the storage unit 11, or it may be downloaded from another download server. Furthermore, the image data that the image generation model M2 inputs and outputs is not limited to still images such as photographs, but also includes moving images.

[0078] Figure 17 is an overview diagram of the image generation model M2. The image generation model M2 is trained to accept both image and text inputs and output a generated training image. In the example shown in Figure 17, the image generation model M2 utilizes so-called generative AI using an architecture such as Stable Diffusion, and includes an image encoder and a text encoder in a VAE (Variational Auto-Encoder), a decoder in the VAE that converts the features output from each encoder back into an image, and an extension network in between. The image generation model M2 converts a given image into an image that conforms to the settings expressed in text. For example, the image generation model M2 is trained to output an image that converts the shooting environment of a given field image into an image taken in a different shooting environment expressed in text. By using the image generation model M2, the processing unit 10 can change the direction of shadows, the color of the sky, etc.

[0079] The image generation model M2 may be a model that uses an autoencoder for image conversion to convert a given on-site image I1 into an image taken in the same environment as the other image, given the on-site image I1 and another image. In this case, the processing unit 10 can also obtain training images taken at different times of day from the image generation model M2 by providing the image generation model M2 with the on-site image I1 and another on-site image I1 taken at the same location in the morning, at noon, or in the evening.

[0080] The image generation model M2 may be a multimodal model that includes a language model provided by an external server. The image generation model M2 may also be a model that transforms images using a generative AI employing a different architecture.

[0081] In the second embodiment, the information processing device 1 performs recognition processing on the on-site image I1 acquired from the terminal device 2 and processing to accept correction of the range of dangerous areas at the terminal device 2, in accordance with the processing procedures shown in Figures 5 and 6, similar to the first embodiment. In the second embodiment, the information processing device 1 performs retraining in step S108 using the image generation model M2.

[0082] Figure 18 is a flowchart showing an example of the retraining process in the second embodiment. The processing procedure shown in Figure 18 corresponds to the detailed processing procedure of step S108 in the processing procedure shown in Figures 5 and 6, which is executed by the information processing device 1 of the second embodiment. For the processing procedure shown in Figure 18 that is common with the processing procedure shown in Figure 7 of the first embodiment, the same step number is used and a detailed explanation is omitted.

[0083] In the second embodiment, when the processing unit 10 acquires a set of field image training data and corrected data (S801), it provides the field image training data and text indicating an instruction to transform the viewpoint from above to the side, or from the side to above, to the image generation model M2 (step S811). The processing unit 10 acquires the image output from the image generation model M2 as a training image (step S812).

[0084] The processing unit 10 provides the image generation model M2 with field images as training data and text indicating instructions to change the weather conditions of the shooting environment (step S813). The processing unit 10 acquires the images output from the image generation model M2 as training images (step S814).

[0085] The processing unit 10 generates a training image by rotating the generated image output from the image generation model M2 in step S812 or step S814 (step S815). The processing in step S815 is the same as the processing in step S802 shown in Figure 7, so a detailed explanation is omitted.

[0086] The processing unit 10 generates a training image by changing the brightness of the generated image output from the image generation model M2 in step S812 or step S814 (step S816). The processing in step S816 is the same as the processing in step S803 shown in Figure 7, so a detailed explanation is omitted.

[0087] In step S812 or step S814, the processing unit 10 generates a training image by overlaying a semi-transparent, solid-color rectangle onto the generated image output from the image generation model M2, changing the color and / or transparency (step S817). The processing in step S817 is the same as the processing in step S804 shown in Figure 7, so a detailed explanation is omitted.

[0088] In step S812 or step S814, the processing unit 10 generates an image as a training image by enlarging a portion of the generated image output from the image generation model M2 (step S818).

[0089] The processing unit 10 stores each of the training images generated in steps S812 and S814-S818 as new training data by combining them with the correction data acquired in step S801 (step S819).

[0090] It is not necessary to perform all of steps S812-S818 on a single field image of training data. The processing unit 10 may perform only steps S811 and S812, or only steps S813 and S814. The processing unit 10 may generate training images by performing all four steps S815-S818 on these two patterns of training images, or it may generate training images in various combinations by performing only three, two, or one of the processes.

[0091] The processing unit 10 updates the parameters of the learning model M1 using the training data stored in step S819 (step S807), and then terminates the retraining process.

[0092] In the second embodiment as well, the processing unit 10 of the information processing device 1 may accept a setting of which of the methods in steps S812-S818 will be used to generate the training images, that is, a setting of the type of training images to be generated. The processing unit 10 can also accept a setting of how many training images to generate. When accepting a setting, the processing unit 10 may output the progress of the learning of the learning model M1 and output a setting screen so that the user can check the progress while making the setting.

[0093] In this second embodiment, by generating training images using the image generation model M2, accurate recognition is possible even from a small number of field images I1.

[0094] (Third embodiment) In the third embodiment, learning images are generated by referring to past records. The configuration of the information processing system 100 in the third embodiment is the same as that of the information processing system 100 in the first embodiment, except for a part of the information processing device 1 which will be described later. Therefore, the same reference numerals are used for common components and detailed descriptions are omitted.

[0095] Figure 19 is a block diagram showing the configuration of the information processing device 1 in the third embodiment. In the information processing system 100 of the third embodiment, the information processing device 1 can access records created by workers before, during, or after construction work, which are stored in another storage device 4, via the communication unit 12. The storage device 4 collects construction records (reports) created from terminal devices 2 used by each worker, and a database is constructed. The processing unit 10 can search for and read construction records from the database in the storage device 4 via the communication unit 12.

[0096] The construction records stored in the storage device 4 include the date and time and location of the construction, the identification data of the workers who performed the work, and text or symbols indicating the type of electrical equipment being worked on. If an accident occurred during the construction, the construction record includes text describing the details of the accident, and this text includes an analysis of the hazardous areas. Records that include an accident also include on-site images of the hazardous areas. Preferably, the construction records include text indicating the type of hazardous area for search purposes. In the third embodiment as well, "on-site images" are not limited to still photographs but also include videos.

[0097] In the third embodiment, the information processing device 1 performs recognition or retraining using the construction records stored in this manner.

[0098] Figures 20 and 21 are flowcharts illustrating an example of a processing procedure by the information processing system 100 of the third embodiment. For the processing procedures shown in Figures 20 and 21 that are common to those shown in Figures 5 and 6 of the first embodiment, the same step numbers are used, and detailed explanations are omitted.

[0099] In the third embodiment, the processing unit 20 of the terminal device 2 acquires image data of the site (S201), and then receives construction data including information indicating the location and type of electrical equipment construction that is planned or currently being carried out (step S221). The processing unit 20 then sends a recognition request to the information processing device 1, which includes the acquired image data and the construction data received in step S221 (step S222).

[0100] When the processing unit 10 of the information processing device 1 receives image data along with a recognition request (S101), it executes the processes in steps S102-S104. The processing unit 10 uses the image data and the information on the location and type of construction work included in the construction data included in the recognition request to retrieve past records similar to the target electrical equipment construction work from the database of the storage device 4 (step S111).

[0101] In step S111, the processing unit 10 extracts past construction projects of the same type, and from the extracted records of past construction projects, it extracts a predetermined number of records of projects where the on-site images included in the records have a high degree of similarity to the on-site images received in step S101. The processing unit 10 may extract records of past construction projects that are located within a short distance range, or it may extract records of past construction projects that are in a similar time period.

[0102] The processing unit 10 creates a screen to notify the user of the hazardous area (step S112) based on the image data acquired in step S101, the data indicating the extent of the hazardous area acquired in step S103, and the records of past construction work extracted in step S111. The screen created in step S112 includes an image in which an object representing the extent of the hazardous area based on the data acquired in step S103 is superimposed on the on-site image of the image data acquired in step S101. The screen created in step S112 also includes text or images explaining the hazardous area and the hazard factors in the records of similar past construction work.

[0103] The processing unit 10 sends the screen data of the created screen to the terminal device 2 that made the recognition request in step S101 (step S113).

[0104] The processing unit 20 of terminal device 2 receives screen data (step S223) and displays a screen showing the hazardous areas in the on-site image on the display unit 23 (step S224). On this screen, the processing unit 20 executes the processes from steps S205 to S209, and if a correction is made (S207: YES), the processing unit 20 sends the correction data (S210).

[0105] The processing unit 10 of the information processing device 1 receives the correction data (S106), and stores the received correction data, the on-site image received in step S101, and the records of past construction work acquired in step S111 in the storage unit 11 as training data (S114).

[0106] The processing unit 10 performs a retraining process using training data including corrected data, on-site images, and records of past construction work (step S115), and then terminates the process.

[0107] Figure 22 is a flowchart showing an example of the relearning process in the information processing device 1 of the third embodiment. The processing procedure shown in Figure 22 corresponds to the details of step S115 of the processing procedure shown in Figures 20 and 21. Of the processing procedure shown in Figure 22, steps that are common with the relearning processing procedure in the first embodiment shown in Figure 7 are given the same step numbers and detailed explanations are omitted.

[0108] The processing unit 10 extracts site images from each of the past construction records included in the training data and stores them as training images (step S501). The processing unit 10 determines the extent of the hazardous area corresponding to the extracted site image (step S502). In step S502, the processing unit 10 determines the extent of the hazardous area from the text and objects (rectangles, arrows, etc.) that point to the hazardous area superimposed on the site image. Specifically, the processing unit 10 determines the extent of the hazardous area to be the area enclosed by a rectangle and the area of ​​the object indicated by an arrow. The processing in steps S501 and S502 may be replaced by obtaining the results performed by the operator of the information processing device 1.

[0109] The processing unit 10 stores the pair of the on-site image extracted in step S501 and the data of the range of the hazardous area determined in step S502 as training data (step S503).

[0110] The processing unit 10 then uses training data, including the training image generated in step S501 and other training images, to perform the processing in steps S801-S807. As a result, if the range of hazardous areas for the on-site image is corrected on terminal device 2, it is possible to improve recognition accuracy by retraining the range of hazardous areas for on-site images of similar past records, assuming that there is insufficient training on similar cases.

[0111] In the retraining procedure shown in Figure 22, the processing content shown in the second embodiment (use of image generation model M2) may be executed. In this case, on-site images from similar cases may be provided to the image generation model M2 along with the original on-site image to generate new training images.

[0112] Figure 23 shows an example of a screen displayed on the display unit 23 of the terminal device 2 of the third embodiment. Figure 23 shows an example of an operation screen 230 displayed when the terminal program P2 is launched on the terminal device 2 of the third embodiment. The operation screen 230 in Figure 23 shows the result of recognition processing performed by the information processing device 1 of the third embodiment on the captured on-site image I1. As shown in Figure 23, the on-site image I1 is an image taken before the construction work to run power lines between the houses was carried out. Objects 236 representing the extent of the hazardous area are displayed on the wall between the houses, the area between the eaves of the first floor of the houses, and on the ground in the foreground.

[0113] In the third embodiment, since records of past construction work are referenced, text 237 is displayed next to object 236, as shown in Figure 23, indicating a summary of accidents that occurred during past construction work. The text 237 in the area between the fence and the eaves of the first floor of the house indicates that there was an accident in the past where someone fell through that gap, and the text 237 on the ground in the foreground indicates that there was an accident where a ladder toppled over because the ground was gravel. This increases the likelihood that electrical equipment workers will be able to identify dangerous areas that they may not have noticed themselves. The text 237 may also include a link to information (such as a file name or URL) that directs the user to a screen where they can view records of past construction work, including accidents.

[0114] In this way, by referring to records of past construction work, it is possible to estimate and present hazardous factors for newly taken site images I1 by extracting records of similar construction work. By extracting and presenting past information in this manner, it is possible to notify terminal device 2 that not only the recognition accuracy but also the safety of electrical equipment installation workers will be improved through retraining.

[0115] (Fourth Embodiment) In the fourth embodiment, the learning model M1 is stored as a local model in the terminal device 2, and recognition processing is performed on the terminal device 2 side, i.e., the edge side, thereby improving real-time performance, and the learning model M1 is retrained on the terminal device 2.

[0116] Figure 24 is a schematic diagram of the information processing system 100 of the fourth embodiment, and Figure 25 is a block diagram showing the configuration of the terminal device 2. Among the components of the information processing system 100 of the fourth embodiment, components that are common with the information processing system 100 of the first embodiment are denoted by the same reference numerals, and detailed explanations are omitted.

[0117] In the fourth embodiment, the terminal device 2 is a wearable device worn by a worker during construction. The terminal device 2 includes a mounting unit 27 including a transparent glass portion, an operation unit 28 including buttons and a pointing device, and a control unit 29. The processing unit 20, storage unit 21, and communication unit 22 are mounted on the control unit 29, the display unit 23, audio input / output unit 25, and camera 26 are mounted on the mounting unit, and the operation unit 24 is mounted on the operation unit 28. The display unit 23 is mounted as a transparent display on the glass portion of the mounting unit 27, and the camera 26 is attached to the shaft of the glass portion to capture the field of view in front. The mounting unit 27 may be implemented as a head-mounted display that displays real-world images being captured by the camera 26 internally, rather than being smart glasses with a transparent glass portion. The operation unit 28 may use a gyro sensor and an accelerometer as pointing devices, or it may be equipped with a wheel or a trackball. The mounting unit 27 and the control unit 29, and the operation unit 28 and the control unit 29, can each send and receive signals via wired or wireless means and operate in sync.

[0118] Workers can perform their tasks while understanding hazardous areas in real time by viewing the actual work site or images of the work site through the display unit 23, as well as by viewing the range of hazardous areas displayed on the display unit 23.

[0119] In the fourth embodiment, in order for the terminal device 2 to recognize the range of dangerous areas in real time, the storage unit 21 stores data of the learning model M1 that has been learned and relearned by the information processing device 1. The terminal program P2 stored in the storage unit 21 of the terminal device 2 in the fourth embodiment includes the same functions as the information processing program P1 on the information processing device 1 side shown below.

[0120] Figure 26 is a flowchart showing an example of the recognition process for hazardous areas in the fourth embodiment. In the fourth embodiment, the terminal device 2 performs most of the functions of the information processing device 1. When the terminal program P2 is started, the processing unit 20 of the terminal device 2 acquires the image output from the camera 26 (step S241). It is assumed that the image output from the camera 26 in step S241 is taken at an electrical equipment construction site.

[0121] The processing unit 20 provides the image data of the step-acquired image to the learning model M1 stored in the storage unit 21 (step S242). The processing unit 20 acquires data indicating the range of the dangerous area output from the learning model M1 (step S243).

[0122] Based on the acquired data, the processing unit 20 displays an object representing the area of ​​the hazardous location on the display unit 23, which is the transparent glass portion (step S244). The worker wearing the glass portion can see the object representing the area of ​​the hazardous location superimposed on their field of view, which includes the actual work site seen through the glass.

[0123] The processing unit 20 accepts corrections via the operation unit 24, which includes a pointing device, or the voice input / output unit 25 (step S245). The processing unit 20 accepts text input from the voice input / output unit 25, which describes the hazardous area recognized by the worker (step S246). If there are no corrections, steps S245 and S246 are omitted.

[0124] The processing unit 20 determines whether or not the recognition result has been corrected (step S247). If it is determined that no correction has been made (S247: NO), the processing unit 20 stores in the storage unit 21 the data of the on-site image in which the range of the hazardous area displayed in step S244 has been recognized, the data indicating the range, and the data of the shooting environment, including information on the electrical equipment to be worked on and information on the date and time of shooting (step S248). The processing unit 20 returns to step S241 and continues processing until the terminal program P2 issues a termination instruction.

[0125] If it is determined in step S247 that a correction has been made (S247:YES), the processing unit 20 stores the correction data, the data of the on-site image in which the extent of the hazardous area to be corrected is recognized, and the data of the shooting environment, including information on the electrical equipment to be worked on and information on the date and time of shooting, in the storage unit 21 (step S249). The correction data is data indicating the extent of the hazardous area and / or text describing the hazardous area and the hazard factors in that hazardous area.

[0126] The processing unit 20 stores the corrected data and the field image data as training data in the storage unit 21 (step S250). The processing unit 20 uses the training data, including the corrected data and the field image, to perform a retraining process on the local learning model (local model) M1 stored in the storage unit 21 (step S251), transmits the data of the learning model M1 after the retraining process to the information processing device 1 (step S252), and terminates the process.

[0127] The information processing device 1 updates the learning model stored in the memory unit 11 using data of the learning model M1 that has been locally retrained at each terminal device 2 and transmitted from terminal device 2 and other terminal devices 2. The learning model M1 updated by the information processing device 1 is then distributed back to terminal devices 2, and distributed associative learning, which repeats this process, may be performed.

[0128] As shown in the processing procedure in Figure 26, edge computing, which uses the learning model M1 to perform recognition processing on terminal device 2, enables real-time notification of hazardous locations and their causes overlaid on the worker's real-world field of view via smart glasses worn by the worker. By notifying workers of hazardous locations they may not have noticed themselves, it is expected that safety in electrical equipment installation work will be further improved.

[0129] Using the voice recognition of the voice input / output unit 25, the terminal device 2 may send an instruction to the information processing device 1 to search for similar past accidents from the database of the storage device 4, which stores past cases as described in the third embodiment, in accordance with the verbal instructions of the worker. In this case, the information processing device 1 implements a language model that uses the storage device 4 as knowledge to create response sentences in response to the instructions. This makes it possible to make the worker aware of the extent of the hazardous area in real time while referring to past cases.

[0130] The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are included. [Explanation of Symbols]

[0131] 1. Information Processing Device 10 Processing Unit 11 Storage section P1 Information Processing Program I1 On-site images I2, I12, I13 Training images 2 Terminal devices 20 Processing Units 23 Display section 26 Cameras P2 Terminal Program

Claims

1. We acquire on-site images during electrical equipment installation work. Using a learning model that is trained to output the recognition result of hazardous areas in an image when an image is input, the recognition result for the acquired on-site image is output. Correction data for the aforementioned recognition result is obtained, The learning model is retrained based on the corrected data and multiple training images generated from the acquired on-site images. Information processing methods.

2. The training images obtained by rotating the acquired on-site images so that the road surface, floor surface, or ground boundary, or the straight line along the vertical direction of the building, aligns with the vertical or horizontal direction in the image, are used for retraining. The information processing method according to claim 1.

3. The acquired on-site images are used as training images by changing their brightness or saturation, The aforementioned on-site image is overlaid with a semi-transparent, solid-color rectangle, and multiple learning images are created by changing the color and / or transparency of the aforementioned solid color. A learning image taken by cropping a portion of the aforementioned on-site image and Use at least one of the following for retraining. The information processing method according to claim 1.

4. Using an image generation model that generates other images based on a given image and given settings, The acquired on-site images are provided to the image generation model along with the settings to generate training images with different shooting environments. The generated training images are used for the retraining process. The information processing method according to claim 1.

5. The image generation model is given a setting to either change the viewpoint from above to the side, or change the viewpoint from the side to above. The information processing method according to claim 4.

6. The image generation model is given a setting to change the weather conditions of the shooting environment. The information processing method according to claim 4.

7. Further training images are generated by rotating the training images generated by the aforementioned image generation model so that the boundaries of road surfaces, floor surfaces, or ground, or the straight lines along the vertical direction of buildings depicted, are aligned with the vertical or horizontal direction in the image. Furthermore, the generated training images are used for retraining. The information processing method according to any one of claims 4 to 6.

8. Based on images taken at past electrical equipment construction sites similar to the site where the acquired site image was taken, and hazard factors estimated from the records of the electrical equipment construction, the training image is generated. The generated training images are used for the retraining process. The information processing method according to any one of claims 1 to 6.

9. The number of training images to be generated, or the type of training images, is accepted according to the learning progress of the learning model. The information processing method according to any one of claims 1 to 6.

10. As a result of the recognition, an image is output in which the area of ​​the hazardous factors is superimposed on the on-site image. On the aforementioned image, the system accepts the modification of the area of ​​the hazardous location and the input of the attributes of the hazardous location. The scope of the corrected hazardous area and the attributes of the said hazardous area are obtained as the correction data. The information processing method according to any one of claims 1 to 6.

11. The recognition results are output sequentially to the on-site images taken before and after the electrical equipment installation work, The system accepts corrections to the recognition results on the monitor screen of the aforementioned on-site image. The information processing method according to any one of claims 1 to 6.

12. We acquire on-site images during electrical equipment installation work. Using a learning model that is trained to output the recognition result of hazardous areas in an image when an image is input, the recognition result for the acquired on-site image is output. Correction data for the aforementioned recognition result is obtained, The learning model is retrained based on the corrected data and multiple training images generated from the acquired on-site images. An information processing device equipped with a processing unit.

13. This includes multiple terminal devices equipped with cameras for capturing on-site images during electrical equipment installation work. Each of the aforementioned terminal devices is: A learning model that has been trained to output the recognition result of dangerous areas in an image when an image is input is used, and the on-site image captured by the camera is given to the learning model. The screen, which includes the aforementioned on-site image and the recognition results output from the learning model, is displayed on the display unit. The aforementioned screen accepts corrections to the recognition result. The learning model is retrained based on the corrected data, which includes data indicating the corrected hazardous areas, and multiple training images generated from the aforementioned on-site images. Information processing system.

14. Each of the aforementioned terminal devices is: The aforementioned learning model is then retrained using on-site images captured by the camera and correction data for accepted modifications. The retrained model is sent to the information processing device connected via communication, The aforementioned information processing device is The integrated processing is performed using the learning model data transmitted from each terminal device. The information processing system according to claim 13.

15. On the computer, We acquire on-site images during electrical equipment installation work. Using a learning model that is trained to output the recognition result of hazardous areas in an image when an image is input, the recognition result for the acquired on-site image is output. Correction data for the aforementioned recognition result is obtained, The learning model is retrained based on the corrected data and multiple training images generated from the acquired on-site images. A computer program that executes a process.

Images