Data collection device and method, and storage medium

Abstract

The embodiment of the invention relates to a data collection device and method, and a storage medium. Provided are a data collection device, a method, and a storage medium that improve the collection accuracy of job logs. A data collection device according to an embodiment includes a road sign detection unit, a position setting unit, an event detection unit, and a data collection unit. The road sign detection unit detects a road sign from an input image. The position setting unit sets a log collection range or an event detection position on the basis of a relative position with respect to the road sign. The event detection unit detects an event at the event detection position. When the event is detected, the data collection unit collects data in the log collection range.

Description

[0001] This application is based on Japanese Patent Application No. 2024-037535 (filing date: March 11, 2024), and claims the benefit of priority from that application, the entire contents of which are incorporated herein by reference. Technical Field

[0002] Embodiments of the present invention relate to a data collection device, method, and storage medium. Background Art

[0003] In heavy particle beam therapy using a heavy particle beam therapy device, accurate aiming of the heavy particle beam directed at the affected area is crucial to avoid damaging normal tissue. Therefore, the user adjusts the beam before treatment so that it is delivered to the center of the treatment room. Electromagnets located at various locations along the path from the accelerator to the treatment room control the beam's position. The beam, accelerated by the accelerator, is delivered to the treatment room via a beam delivery tube. The user confirms the beam's position as it passes through the locations where the electromagnets are located and adjusts the parameters of each electromagnet so that the beam passes through the center of the delivery tube.

[0004] Typically, electromagnets are installed in more than ten locations, requiring users to visually adjust dozens of parameters. Furthermore, adjustments must be made for each beam intensity and angle. For example, if the beam intensity can be adjusted in 600 steps and the beam angle can be adjusted in 360 steps, this requires 600 x 360 = 216,000 adjustments.

[0005] To reduce the burden on users, technologies that automate beam adjustment are desired. Automating beam adjustment requires collecting logs of actual beam adjustment operations. However, applications for adjusting electromagnet parameters and confirming beam positions are generally separate, and these applications often lack pre-installed functionality for collecting operation logs. Consequently, effectively collecting user operation logs is difficult.

[0006] Furthermore, to automate beam adjustment, there are data collection devices that collect actual work data performed on a work terminal. For example, such data collection devices acquire images displayed on a work terminal's display and analyze the acquired images to collect work data. This allows actual work data to be collected without having to change the work terminal.

[0007] In such data collection devices, smooth data collection can sometimes be hindered if user intervention occurs during data collection. For example, image analysis accuracy can be reduced if the position or layout of windows changes during operation. Furthermore, because the user must determine the timing of data collection, collection accuracy can vary depending on user interaction. Summary of the Invention

[0008] The problem to be solved by the embodiments is to provide a data collection device, method, and storage medium that improve the accuracy of job log collection.

[0009] To address this issue, a data collection device according to an embodiment includes a landmark detection unit, a position setting unit, an event detection unit, and a data collection unit. The landmark detection unit detects landmarks from an input image. The position setting unit sets a log collection range or an event detection position based on the relative position of the landmark. The event detection unit detects an event at the event detection position. When the event is detected, the data collection unit collects data within the log collection range.

[0010] According to the data collection device having the above configuration, it is possible to improve the accuracy of collecting job logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a diagram showing an example of the configuration of the data collection device according to the first embodiment.

[0012] Figure 2 This is a flowchart illustrating the processing procedure of the data collection process according to the first embodiment.

[0013] Figure 3 This is a diagram showing the flow of data in the data collection process according to the first embodiment.

[0014] Figure 4 This is a diagram showing an example of an input image according to the first embodiment.

[0015] Figure 5 yes Figure 4 A diagram showing changes in the position of the display window.

[0016] Figure 6 This is a diagram showing an example of an exceptional image according to the first embodiment.

[0017] Figure 7 This is a diagram showing an example of collected data according to the first embodiment.

[0018] Figure 8 This is a diagram showing an example of an input image according to a first modification of the first embodiment.

[0019] Figure 9 This is a diagram showing an example of an input image according to the second modification of the first embodiment.

[0020] Figure 10This is a diagram showing an example of the configuration of a data collection device according to the second embodiment.

[0021] Figure 11 This is a flowchart illustrating the processing procedure of the data collection process according to the second embodiment.

[0022] Figure 12 This is a diagram showing the flow of data in the data collection process according to the second embodiment.

[0023] (Explanation of Symbols)

[0024] 100: Data collection device; 11: Processing circuit; 12: Storage device; 13: Input device; 14: Communication device; 15: Display device; 111: Signpost detection unit; 112: Position setting unit; 113: Event detection unit; 114: Exception determination unit; 115: Data collection unit; 116: Signpost setting unit; 200: Operation terminal; A1, A2, A3: Input image; B1, B2: Display window; C1, C2: Log collection range; D1, D2: Event detection position; E1, E2: Signpost. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the data collection device, method, and storage medium will be described in detail with reference to the accompanying drawings. In the following description, components having substantially the same function and structure are denoted by the same reference numerals, and repeated description will be given only when necessary.

[0026] (First embodiment)

[0027] Figure 1 This figure shows an example of the configuration of a data collection device 100 according to the first embodiment. The data collection device 100 is connected to a work terminal 200. The connection between the data collection device 100 and the work terminal 200 may be a wired connection, a wireless connection, or a connection via a network.

[0028] The work terminal 200 is a terminal device used by workers to perform predetermined tasks. The work terminal 200 includes input interfaces such as a keyboard and mouse, a console device with processing circuitry, and a display. For example, the work terminal 200 is a typical PC or tablet computer. Workers perform tasks using, for example, a designated application installed on the work terminal 200.

[0029] The work terminal 200 is, for example, a device for adjusting a heavy particle beam in a heavy particle beam therapy system. In this case, the operator activates a parameter input application and a beam observation application displayed on the display of the work terminal 200, and uses the parameter input application to adjust parameters so that the beam distribution displayed on the beam observation application becomes the desired distribution.

[0030] The data collection device 100 collects work data from the work terminal 200. The data collection device 100 continuously acquires the display image of the work terminal 200 to collect the work log of the work terminal 200. The data collection device 100 is, for example, a common laptop PC or tablet computer. The work log is information related to work performed using the work terminal 200. Examples of the work log include operation logs, parameter change histories, and analysis results of displayed images. The work log can also be referred to as work data or collected data. Furthermore, the data collection device 100 can also be referred to as a data collection unit.

[0031] Data collection device 100 is a computer having a processing circuit 11, a storage device 12, an input device 13, a communication device 14, and a display device 15. Data communication between processing circuit 11, storage device 12, input device 13, communication device 14, and display device 15 is performed via a bus. Input device 13 and display device 15 may not be provided.

[0032] The processing circuit 11 includes a processor such as a CPU (Central Processing Unit) and a memory such as RAM (Random Access Memory). The processing circuit 11 includes a landmark detection unit 111, a position setting unit 112, an event detection unit 113, an exception determination unit 114, and a data collection unit 115. The processing circuit 11 executes programs to implement the landmark detection, position setting, event detection, exception determination, and data collection functions of these components.

[0033] The storage device 12 is composed of a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), an integrated circuit storage device, etc. The storage device 12 stores programs and the like.

[0034] The program is stored in a non-temporary computer-readable recording medium such as the storage device 12. The program can be installed as a single program that describes all the functions of the above-mentioned parts, or it can be installed as multiple modules divided into several functional units. In addition, the above-mentioned parts can also be installed using integrated circuits such as application-specific integrated circuits (ASICs). In this case, they can be installed in a single integrated circuit or in multiple integrated circuits.

[0035] The input device 13 inputs various commands from the operator. A keyboard, a mouse, various switches, a touch pad, a touch panel display, etc. can be used as the input device 13. The output signal from the input device 13 is supplied to the processing circuit 11.

[0036] The communication device 14 is an interface for performing data communication with an external device connected to the data collection apparatus 100 via a network. For example, the communication device 14 performs data communication with the work terminal 200.

[0037] The display device 15 displays various information. A CRT (Cathode-Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, an LED (Light-Emitting Diode) display, a plasma display, or any other display known in the art can be appropriately used as the display device 15. Alternatively, the display device 15 may be a projector.

[0038] Next, each function performed by each unit of the processing circuit 11 will be described in detail.

[0039] The road sign detection unit 111 detects road signs from the input image input to the work terminal 200. At this point, the road sign detection unit 111 first continuously acquires image data of the display image displayed on the monitor of the work terminal 200. At this point, the unit may also acquire the display screen displayed on the monitor of the work terminal 200, or acquire output signals output from the console device of the work terminal 200 to the monitor in order to display the display screen. Hereinafter, the image data acquired from the work terminal 200 is referred to as the input image.

[0040] Afterwards, the landmark detection unit 111 detects the position of a pre-specified landmark from the input image. The landmark is used to determine the log collection range and event detection position. It is best that the landmark is a unique pattern in the input image whose positional relationship with the log collection range and event detection position does not change even if the layout of the input image or the position of the application in the input screen changes. A unique pattern refers to a portion with a unique color or a unique shape in the input image. For example, an icon or label with a unique color or a unique shape can be used as a landmark.

[0041] The log collection range is the area within the input image where data is collected and is used to collect job logs. For example, the log collection range is the area within which images are captured. The log collection range can also be referred to as the data collection range or the capture range. For example, the log collection range includes the parameter display area in parameter input applications and the energy distribution display area in beam observation applications.

[0042] An event detection location is a location where events that determine the timing for executing data collection are detected. For example, a display unit displaying a specific operation button or a specific parameter can be used as the event detection location. For example, a condition change event can be detected by the state of an icon of a specific operation button being operated. Alternatively, a parameter adjustment event can be detected by a change in the value of a parameter on a display unit displaying a specific parameter.

[0043] In the detection of road signs, for example, techniques using template matching and filter response can be used. The technique using template matching is a method of maintaining an image of a pre-set road sign as a template and detecting the portion that matches the maintained image from within the screen. The technique using filter response is a method of detecting road signs from within the screen by applying a filter to an image, after the shape of the road sign is known, by pre-designing a filter that reflects the shape of the road sign. Furthermore, multiple road signs can also be detected. For example, road signs used to set the log collection range and road signs used to set the event detection position can also be detected separately.

[0044] The position setting unit 112 sets the log collection range and event detection location based on the location of the landmark detected by the landmark detection unit 111. In this case, the position setting unit 112 sets the log collection range based on the relative location of the landmark, and sets the event detection location based on the relative location of the landmark. The positional relationship between the landmark and the log collection range, and the positional relationship between the landmark and the event detection location, are pre-set by the user and stored in the storage device 12.

[0045] Event detection unit 113 detects events at event detection locations. Events are used to determine the timing of data collection. Examples of events include specific operations and screen transitions caused by specific operations. Examples of event detection methods include detecting image changes and detecting specific image change patterns.

[0046] Methods for detecting image changes can be used when continuously acquiring input images to collect time-series information. For example, the image pattern at the event detection location is constantly stored, and the difference between the immediately preceding and current image patterns is calculated. When the difference exceeds a threshold, it can be determined that an event has occurred. Furthermore, if the image pattern at the event detection location includes numbers or text, the numbers or text can be analyzed and stored, and changes in the numbers or text can be detected as events.

[0047] The method of detecting a specific image pattern can be used to detect an operation that displays a specific image pattern. For example, the image pattern at the event detection location is compared with a pre-stored specific image pattern, and if the image patterns match, it is determined that an event has occurred. For example, a specific button can be set as the event detection location. When the image pattern of the selected button changes to the specific image pattern when the button is selected, the occurrence of the image pattern of the selected button is detected as an event, thereby detecting that the user has pressed the specific button on the display screen.

[0048] The exception determination unit 114 analyzes the image of the log collection range at the time when the event is detected and determines whether the image of the log collection range is an exception image. Exception images are data that has a high probability of being an exception value or a deviation value when the image of the log collection range is analyzed. For example, if other display units are displayed on the log collection range or the data display area of ​​the collection object is blocked by other display units, the image of the log collection range is determined to be an exception image. Other display units are, for example, windows of other applications, dialog boxes for file transfers, drop-down menus, parameter setting GUIs, mouse cursors, etc. Alternatively, image analysis can be used to detect text or numbers within the image of the log collection range. If no text or numbers are displayed, or if abnormal text or numbers are displayed, the image of the log collection range is determined to be an exception image. Alternatively, a reference image can be pre-stored as a reference, and the similarity between the image of the log collection range and the reference image can be calculated. If the similarity is low, the image is determined to be an exception image.

[0049] The data collection unit 115 collects data within the log collection range at the time the event is detected. At this time, the data collection unit 115 removes images determined as exception images by the exception determination unit 114 and collects only images that are not determined as exception images. The collected images are stored in the storage device 12, for example.

[0050] The data collection unit 115 may also collect the results of image analysis of the collected images. For example, the data collection unit 115 may collect characters and numbers obtained through image analysis, or may analyze and collect the positions, distribution, and shapes of objects within the images.

[0051] (Data Collection and Processing)

[0052] Next, the operation of the data collection process executed by the data collection device 100 will be described. Figure 2 This is a flowchart showing an example of the process of data collection processing. Figure 3 This diagram illustrates an example of the flow of data during data collection processing. The processing procedures described below are merely examples and can be modified as appropriate. Furthermore, steps in the processing procedures described below may be omitted, replaced, or added as appropriate depending on the implementation.

[0053] Here, we will describe the collection of job logs for jobs using the beam confirmation application. The work terminal 200 uses the beam observation application to observe the heavy particle beam passing near the electromagnets of a heavy particle beam therapy system. This example uses the data collection device 100 to collect the energy distribution of the heavy particle beam. Data collection is performed while the data collection device 100 is connected to the work terminal 200.

[0054] Figure 4 as well as Figure 5 is a diagram showing an example of an input image A1 obtained from the work terminal 200. The input image A1 is a display image output to the display of the work terminal 200. Figure 4 as well as Figure 5 In the example of FIG. 1 , a display window B1 of the beam observation application is displayed in the input image A1 . Figure 5 The position of the display window B1 is shown from Figure 4 The state changes shown.

[0055] Here, the display area C1 for the beam's energy distribution (hereinafter referred to as the energy distribution display) is set to the log collection range, and the display area D1 for the beam's analysis results (hereinafter referred to as the analysis result display) is set to the event detection location. The energy distribution display C1 displays the energy distribution of the beam passing near the selected electromagnet. The analysis result display D1 displays the position coordinates and size of the beam's center of gravity as part of the analysis results displayed on the energy distribution display C1. For example, the beam's horizontal and vertical dimensions are displayed. The display on the analysis result display D1 is updated each time the beam in the energy distribution display C1 is updated.

[0056] In addition, a specific icon E1 in the beam observation application is set as a landmark. The icon E1 used as a landmark has a unique shape different from other parts in the display window B1. The icon E1 is, for example, a camera icon. Figure 4 as well as Figure 5 As shown, even if the position of the display window B1 of the beam observation application on the display screen A changes, the relative position of the icon E1 with respect to the energy distribution display portion C1 and the analysis result display portion D1 does not change.

[0057] (Step S101)

[0058] After the data collection process starts, the road sign detection unit 111 continuously acquires image data of the input image A1 displayed on the display of the work terminal 200 from the work terminal 200. The acquired image data is used as the input image.

[0059] (Step S102)

[0060] The landmark detection unit 111 detects the icon E1 set as a landmark by performing image analysis on the input image.

[0061] (Step S103)

[0062] Next, the position setting unit 112 reads the relative positions of the energy distribution display unit C1 and the analysis result display unit D1 with respect to the icon E1 and determines the positions of the energy distribution display unit C1 and the analysis result display unit D1 based on the position of the icon E1 detected in step S102 and the read relative positions.

[0063] Thereafter, the position setting unit 112 sets the energy distribution display unit C1 as the log collection range and sets the analysis result display unit D1 as the event detection position.

[0064] (Step S104)

[0065] Next, the event detection unit 113 continuously performs image analysis on the analysis result display unit D1, which is set as the event detection position, and determines the presence or absence of an event based on the analysis results. If the coordinates or size of the center of gravity of the beam displayed on the analysis result display unit D1 changes due to a change in parameters related to the electromagnet, the event detection unit 113 detects the change in the beam as an event.

[0066] (Step S105)

[0067] When an event is detected (step S104 - "Yes"), the exception determination unit 114 determines whether the image in the energy distribution display unit C1 set as the log collection range is an exception image with respect to the input image where the event is detected. Figure 6As shown, when a display window F of another application is superimposed on the energy distribution display portion C1 , the exception determination unit 114 determines that the image of the energy distribution display portion C1 is an exception image that is inappropriate as collected data.

[0068] (Step S106)

[0069] When an event is detected (step S104-"Yes") and the image of the energy distribution display part C1 in the input image where the event is detected is not an exception image (step S105-"No"), the data collection unit 115 saves the image of the energy distribution display part C1 in the input image where the event is detected as data in the log collection range.

[0070] The data collection unit 115 also performs image analysis on the stored image of the energy distribution display unit C1 to calculate the center of gravity and size of the beam. The data collection unit 115 also stores the analysis results for each input image. Figure 7 : is a diagram showing the state of the analysis result of the image stored in the energy distribution display unit C1. Figure 7 In the image data, for each event that is not determined to be an exceptional image, the X-axis coordinate "x" of the center of gravity of the beam, the Y-axis coordinate "y" of the center of gravity of the beam, the size of the beam in the X-axis "sx", and the size of the beam in the Y-axis "sy" are stored. Figure 7 In , the above analysis results (x, y, sx, sy) are saved for each electromagnet adjusted.

[0071] (Step S107)

[0072] The processing circuit 11 repeatedly executes the above-described steps S101 through S106 until the operation using the beam observation application in the operation terminal 200 ends (step S107 - No), continuously collecting images of the energy distribution display unit C1 at the time of the event as operation data. Furthermore, when the operation using the beam observation application ends (step S107 - No), the data collection device 100 in the processing circuit 11 terminates the series of data collection processes.

[0073] The effects of the data collection device 100 according to this embodiment will be described below.

[0074] If the work terminal 200 performing beam adjustment is a highly versatile device such as a PC, the location on the screen where the work log should be collected and the location of interest for detecting screen transitions may change depending on the user's operation. Furthermore, the display window of the collection target may be obscured by the display window of another application. In these cases, data collection may fail. Furthermore, while it is preferable to use numerical values ​​or beam shapes that change with changing conditions as event detection locations to determine data collection timing, these numerical values ​​and beam shapes are constantly changing, making direct detection difficult through image analysis.

[0075] The data collection device 100 according to this embodiment is connected to a work terminal 200 equipped with a display. The data collection device 100 includes a landmark detection unit 111, a position setting unit 112, an event detection unit 113, an exception determination unit 114, and a data collection unit 115. The landmark detection unit 111 detects landmarks from an input image. The input image is the display image displayed on the display of the work terminal 200. The input image includes a display window that displays the log collection range, event detection locations, and landmarks. The display window may be, for example, a display window for an application for parameter input of a heavy particle beam therapy system or a display window for a beam observation application. The log collection range includes, for example, a display area for operation logs such as adjustment objects and parameters. An event may be, for example, a change in the log collection range. The relative position of the landmark with respect to the log collection range and event detection location does not change depending on the position of the display window in the input image. In other words, a portion of the landmark that does not change relative to the log collection range and event detection location even if the position of the application's display window changes can be used. For example, a portion having a unique color or shape in an input image is used as a road sign.

[0076] The position setting unit 112 sets the log collection range and event detection location based on the relative position to the road sign. The event detection unit 113 detects an event at the event detection location. Upon detecting an event, the data collection unit 115 collects image data within the log collection range. At this point, the data collection unit 115 may also perform image analysis on the log collection range and collect the analysis results.

[0077] With the above-described structure, the data collection device 100 according to this embodiment can detect landmarks from the input image output by the operation terminal 200 and use the relative position of the landmarks to determine the log collection range and event detection position. Therefore, even if the layout or position of the application changes due to user operations, the log collection range and event detection position can be accurately determined, allowing efficient collection of operation logs. Furthermore, by using unique colors and shapes as landmarks, landmarks can be easily detected from the input image.

[0078] The data collection device 100 according to this embodiment further includes an exception determination unit 114 that performs exception determination on image data within the log collection range. The data collection unit 115 collects only image data that has not been determined to be an exception. This configuration allows for exception determination for images within the log collection range, eliminating images where the log collection range is obscured by other displays or where the log collection range was incorrectly set, and allows job logs to be collected. This improves the accuracy of job log collection. Furthermore, by eliminating unnecessary job logs, the data storage capacity and costs can be reduced.

[0079] Furthermore, the log collection range and positional relationship of the event detection location for a landmark may change depending on the display window magnification. In this case, the display window magnification can be determined based on the size of the detected landmark, and the relative position can be corrected based on the magnification.

[0080] Alternatively, it is also possible to use a signpost to set only the log collection range. In this case, compared to setting the log collection range without using a signpost, the area that suppresses unintended image mixing is set as the log collection range, which can improve the collection accuracy of the job log. Therefore, it is possible to achieve a reduction in the capacity of stored data and a reduction in costs. Alternatively, it is also possible to use a signpost to set only the event detection position. In this case, compared to setting the event detection position without using a signpost, changes in the log collection range can be effectively detected, which can improve the collection accuracy of the job log. Therefore, it is possible to achieve a reduction in the capacity of stored data and a reduction in costs.

[0081] That is, according to the data collection device 100 involved in this embodiment, the collection accuracy of the job log can be improved by setting the detection position (log collection range or event detection position) for determining the collection range (collection position or collection timing) of the job log based on the relative position with respect to the signpost.

[0082] (First Modification)

[0083] The above embodiment primarily describes the collection of job logs for a beam observation application. This variation describes the collection of job logs for a parameter input application. In this variation, the operation terminal 200 is used to modify the parameters of the electromagnets in a heavy particle beam therapy system using the parameter input application, and the data collection device 100 is used to collect the user-entered parameters.

[0084] Figure 8 : is a diagram showing an example of an input image A2 output to the display of the work terminal 200 in this modification. Figure 8 In the example of FIG. 1 , a display window B2 of the parameter input application is displayed in the input image A2 .

[0085] Here, the parameter input section C2 for inputting parameters is set to the log collection range, and the parameter change button D2 is set to the event detection position. The parameter input section C2 displays the set values ​​of the parameters related to the selected electromagnet. The parameters displayed in the parameter input section C2 can be changed by user input. The operation button D2 is a setting button for setting the parameters entered in the parameter input section C2. When the operation button D2 is selected, the current parameters are updated to the parameters displayed in the parameter input section C2, and the color or shape of the operation button D2 changes.

[0086] In addition, a specific label E2 in the parameter input application is set as a signpost. The label E2 used as a signpost has a unique color different from other parts in the display window B2. For example, only the label E2 is displayed in black in the display window B2. Figure 8 As shown, even if the position of the display window B2 of the parameter input application on the display screen A changes, the relative position of the label E2 with respect to the parameter input portion C2 and the operation button D2 does not change.

[0087] In this variation, during step S104, event detection unit 113 continuously analyzes the image of operation button D2, which is set as the event detection position, and determines whether an event has occurred based on the analysis results. When operation button D2 changes to a state indicating that it has been operated, event detection unit 113 determines that the parameters related to the electromagnet have been updated and detects the parameter update as an event.

[0088] Furthermore, during step S106, the data collection unit 115 stores the image of the parameter input section C2 in the input image where the event was detected as data within the log collection range. At this point, the data collection unit 115 performs image analysis on the stored image of the parameter input section C2 to detect parameters and stores the analysis results for each input image. This allows the collection of updated parameters each time the operation button D2 (which serves as a setting button) is selected and the electromagnet's parameters are updated.

[0089] (Second Modification)

[0090] This variation describes the collection of operation logs for both the beam confirmation application and the parameter input application. In this variation, a worker uses a work terminal 200 to modify the parameters of each electromagnet in the heavy particle beam therapy system using the parameter input application, and uses the beam observation application to observe the heavy particle beam passing near each electromagnet in the heavy particle beam therapy system. The data collection device 100 collects the parameters set using the parameter input application and the energy distribution of the heavy particle beam when the parameters are updated to those values.

[0091] Figure 9 : is a diagram showing an example of an input image A3 output to the display of the work terminal 200 in this modification. Figure 9 In the example of FIG, in the input image A3, the display window B1 of the observation application and the display window B2 of the parameter input application are displayed. Figure 4 as well as Figure 5 The display window B2 for parameter input is the same as that in Figure 8 The display windows are the same as those described in , so detailed description is omitted.

[0092] In this variation, in step S102, landmarks are detected from input image A3. At this point, the landmark detection unit 111, based on the fact that the icon E1 and label E2 are detected as landmarks, determines that both the parameter input application and the beam confirmation application are displayed. Subsequently, in step S103, the energy distribution display unit C1 and parameter input unit C2 are set as the log collection range, and the analysis result display unit D1 and operation button D2 are set as the event detection location.

[0093] Then, in step S104, if the operation button D2 changes to a state indicating that it has been operated, it is determined that an event has occurred. In this case, in step S106, the data collection unit 115 saves the image of the parameter input unit C2 and the parameter setting value obtained by image analysis.

[0094] Furthermore, when the operation button D2 is pressed and the parameters are changed, the voltage of the electromagnet gradually changes, gradually changing the energy distribution display C1 in the beam confirmation application, and the coordinates and size of the beam's center of gravity displayed on the analysis result display D1 change. In step S104, if the coordinates and size of the beam's center of gravity displayed on the analysis result display D1 change, it is determined that an event has occurred. In step S106, the image of the energy distribution display C1 and the position and size of the heavy particle beam obtained through image analysis are saved. At this time, the data collection unit 115 associates the position and size of the heavy particle beam obtained from the energy distribution display C1 with the parameter settings obtained from the parameter input unit C2 and stores them.

[0095] In addition, when it is determined in step S105 that the image of the energy distribution display portion C1 or the image of the parameter input portion C2 is an exceptional image, the image is not stored.

[0096] According to this modification, every time the parameters of the electromagnet are updated, the position and size of the heavy particle beam can be associated with the parameters and stored.

[0097] Alternatively, the timing of an event may be detected in one of the beam confirmation application and the parameter input application, and data may be collected simultaneously in both applications. In this case, event detection may be performed in only one of the beam confirmation application and the parameter input application.

[0098] (Second embodiment)

[0099] The second embodiment is described. This embodiment is a modification of the structure of the first embodiment as follows. The description of the same structure, operation, and effects as the first embodiment is omitted. In the first embodiment, pre-set landmarks were used. In this embodiment, the landmarks are automatically set using an input image.

[0100] Figure 10 1 is a diagram showing a configuration example of a data collection device 100 according to this embodiment. Figure 10 As shown, the processing circuit 11 further includes a roadmap setting unit 116. The processing circuit 11 also realizes a roadmap setting function by executing a program.

[0101] The landmark setting unit 116 automatically sets landmarks based on an image for landmark setting (hereinafter referred to as a setting image). The setting image is an image of the same type as the input image, for example, input by the user. The landmark setting unit 116 detects a unique pattern within the input setting image and automatically sets the detected unique pattern as a landmark. Alternatively, a group of images included in the animation data may be used as the setting image.

[0102] For example, a landmark can be set by exploiting the low correlation between a unique pattern and other areas within the input image. In this case, the landmark setting unit 116 averages the inverse of the correlation with other locations for each pixel in the setting image and calculates the result as the degree of uniqueness. Regions where pixels with high degrees of uniqueness are concentrated are detected as unique patterns, and the detected regions are set as landmarks.

[0103] Alternatively, an area detected as the unique pattern and whose relative position relative to the designated log collection range and event detection location remains unchanged may be set as a landmark. In this case, the user inputs multiple setting images in which the position of the application display window on the image changes, and specifies the log collection range and event detection location in each setting image. The landmark setting unit 116 detects an area in each setting image whose positional relationship with the log collection range and event detection location remains unchanged and has a high degree of uniqueness, and sets the detected area as the landmark.

[0104] (Data Collection and Processing)

[0105] Next, the operation of the data collection process executed by the data collection device 100 according to this embodiment will be described. Figure 11 This is a flowchart showing an example of the process of data collection processing. Figure 12 This is a diagram showing an example of the flow of data in the data collection process.

[0106] In this variation, first, the user inputs a setting image into the data collection device 100. The landmark setting unit 116 obtains the input setting image (step S201), performs image analysis on the setting image, detects a unique pattern in the setting image, and sets the detected unique pattern as a landmark (step S202).

[0107] After that, through the processing of step S203 to step S209, the operation log of the operation terminal 200 can be collected using the automatically set roadmap. Figure 2 The processing of steps S101 to S107 is the same, so detailed description is omitted.

[0108] According to the data collection device 100 involved in this embodiment, it is unnecessary to set icons and labels suitable for landmarks in advance, and operation logs can be collected using automatically set landmarks, which can reduce the burden on users.

[0109] As described above, according to any one of the above-described embodiments, it is possible to provide a data collection device, method, and storage medium that improve the accuracy of job log collection.

[0110] While several embodiments of the present invention have been described, these embodiments are merely illustrative and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their variations are intended to be within the scope and spirit of the invention, and are also intended to be within the scope of the invention described in the claims and their equivalents.

[0111] Furthermore, the above-mentioned embodiments can be summarized as the following technical solutions.

[0112] (Technical Solution 1)

[0113] A data collection device comprising:

[0114] a road sign detection unit, detecting road signs from an input image;

[0115] a position setting unit for setting a log collection range or an event detection position according to a relative position to the landmark;

[0116] an event detecting unit that detects an event at the event detection position; and

[0117] The data collection unit collects data within the log collection range when the event is detected.

[0118] (Technical Solution 2)

[0119] According to technical solution 1,

[0120] further comprising an exception determination unit for executing exception value determination on the image in the log collection range,

[0121] The data collection unit collects only images that are not determined to be exceptional values.

[0122] (Technical Solution 3)

[0123] According to technical solution 1,

[0124] The input image includes a display window displaying the log collection range, the event detection location, and the road sign.

[0125] The relative position of the landmark with respect to the log collection range and the event detection position does not change according to the position of the display window in the input image.

[0126] (Technical Solution 4)

[0127] According to technical solution 3,

[0128] The road sign is a unique pattern having a specific color or shape in the input image.

[0129] (Technical Solution 5)

[0130] According to technical solution 1,

[0131] The log collection range includes the display area of ​​the job log,

[0132] The event is a change in the log collection scope.

[0133] (Technical Solution 6)

[0134] According to technical solution 1,

[0135] The data collection unit performs image analysis on the log collection range and collects analysis results.

[0136] (Technical Solution 7)

[0137] According to technical solution 1,

[0138] The data collection device is connected to a work terminal with a display.

[0139] The input image is a display image displayed on the display of the work terminal.

[0140] (Technical Solution 8)

[0141] According to technical solution 1,

[0142] It also includes a landmark setting unit that sets the landmark based on the input image.

[0143] (Technical Solution 9)

[0144] A data collection device comprising:

[0145] a landmark detecting unit for detecting a landmark from an input image obtained from a heavy particle beam therapy apparatus that irradiates heavy particle beams using an electromagnet;

[0146] a position setting unit that sets, based on a relative position with respect to the landmark, an area displaying an energy distribution of the heavy particle beam near the electromagnet as a log collection range and an area displaying an analysis result of the heavy particle beam as an event detection position;

[0147] an event detection unit that detects a change in the analysis result at the event detection position as an event; and

[0148] The data collection unit analyzes the energy distribution of the heavy particle beam in the log collection range when the event is detected, and collects the analysis result as data in the log collection range.

[0149] (Technical Solution 10)

[0150] A data collection device comprising:

[0151] a landmark detecting unit for detecting a landmark from an input image obtained from a heavy particle beam therapy apparatus that irradiates heavy particle beams using an electromagnet;

[0152] a position setting unit that sets an area displaying parameters related to the electromagnet as a log collection range based on a relative position with respect to the road sign, and sets an operation unit operated when changing the parameters as an event detection position;

[0153] an event detecting section that detects a change of the operating section at the event detection position as an event; and

[0154] The data collection unit collects the parameters displayed in the log collection range as data in the log collection range when the event is detected.

[0155] (Technical Solution 11)

[0156] According to technical solution 9 or 10,

[0157] further comprising an exception determination unit for executing exception value determination on the image in the log collection range,

[0158] The data collection unit collects only images that are not determined to be exceptional values.

[0159] (Technical Solution 12)

[0160] According to technical solution 9 or 10,

[0161] The input image includes a display window displaying the log collection range, the event detection location, and the road sign.

[0162] The relative position of the landmark with respect to the log collection range and the event detection position does not change according to the position of the display window in the input image.

[0163] (Technical Solution 13)

[0164] According to technical solution 12,

[0165] The road sign is a unique pattern having a specific color or shape in the input image.

[0166] (Technical Solution 14)

[0167] A method having:

[0168] The road sign detection unit detects road signs from the input image;

[0169] The position setting unit sets a log collection range or an event detection position according to a relative position to the landmark;

[0170] an event detecting section detecting an event at the event detection position; and

[0171] When the event is detected, the data collection unit collects image data in the log collection range.

[0172] (Technical Solution 15)

[0173] A computer-readable non-transitory storage medium storing a program for causing a computer to:

[0174] Function to detect road signs from input image;

[0175] A function of setting a log collection range or an event detection location according to a relative position to the landmark;

[0176] a function of detecting an event at the event detection location; and

[0177] A function of collecting image data of the log collection range when the event is detected.

Claims

1. A data collection device comprising: a road sign detection unit, detecting road signs from an input image; a position setting unit for setting a log collection range or an event detection position according to a relative position to the landmark; an event detection unit configured to detect an event at the event detection position; as well as The data collection unit collects data within the log collection range when the event is detected.

2. The data collection device according to claim 1, wherein: further comprising an exception determination unit for executing exception value determination on the image in the log collection range, The data collection unit collects only images that are not determined to be exceptional values.

3. The data collection device according to claim 1, wherein: The input image includes a display window displaying the log collection range, the event detection location, and the road sign. The relative position of the landmark with respect to the log collection range and the event detection position does not change according to the position of the display window in the input image.

4. The data collection device according to claim 3, wherein: The road sign is a unique pattern having a specific color or shape in the input image.

5. The data collection device according to claim 1, wherein: The log collection range includes the display area of ​​the job log, The event is a change in the log collection scope.

6. The data collection device according to claim 1, wherein: The data collection unit performs image analysis on the log collection range and collects analysis results.

7. The data collection device according to claim 1, wherein: The data collection device is connected to a work terminal with a display. The input image is a display image displayed on the display of the work terminal.

8. The data collection device according to claim 1, wherein: It also includes a landmark setting unit that sets the landmark based on the input image.

9. A data collection device comprising: a landmark detecting unit for detecting a landmark from an input image obtained from a heavy particle beam therapy apparatus that irradiates heavy particle beams using an electromagnet; a position setting unit that sets, based on a relative position with respect to the landmark, an area displaying an energy distribution of the heavy particle beam near the electromagnet as a log collection range and an area displaying an analysis result of the heavy particle beam as an event detection position; an event detection unit that detects a change in the analysis result at the event detection position as an event; as well as The data collection unit analyzes the energy distribution of the heavy particle beam in the log collection range when the event is detected, and collects the analysis result as data in the log collection range.

10. A data collection device comprising: a landmark detecting unit for detecting a landmark from an input image obtained from a heavy particle beam therapy apparatus that irradiates heavy particle beams using an electromagnet; a position setting unit that sets an area displaying parameters related to the electromagnet as a log collection range based on a relative position with respect to the road sign, and sets an operation unit operated when changing the parameters as an event detection position; an event detecting unit configured to detect a change of the operating unit at the event detection position as an event; as well as The data collection unit collects the parameters displayed in the log collection range as data in the log collection range when the event is detected.

11. The data collection device according to claim 9 or 10, wherein: further comprising an exception determination unit for executing exception value determination on the image in the log collection range, The data collection unit collects only images that are not determined to be exceptional values.

12. The data collection device according to claim 9 or 10, wherein: The input image includes a display window displaying the log collection range, the event detection location, and the road sign. The relative position of the landmark with respect to the log collection range and the event detection position does not change according to the position of the display window in the input image.

13. The data collection device according to claim 12, wherein: The road sign is a unique pattern having a specific color or shape in the input image.

14. A method comprising: The road sign detection unit detects road signs from the input image; The position setting unit sets a log collection range or an event detection position according to a relative position to the landmark; An event detection unit detects an event at the event detection position; as well as When the event is detected, the data collection unit collects image data in the log collection range.

15. A computer-readable non-transitory storage medium storing a program for causing a computer to: Function to detect road signs from input image; A function of setting a log collection range or an event detection location according to a relative position to the landmark; a function of detecting an event at the event detection location; as well as A function of collecting image data of the log collection range when the event is detected.

Images