Information processing device, information processing method, and computer program

Abstract

To provide a computer program for reducing a load on a meter reading work to read an analog meter.SOLUTION: A computer program allows a computer to function as: a mode selection section for selecting one of a first mode to recognize a pointer of a reading object by detecting the pointer based on image data obtained by photographing an analog meter and a second mode to recognize the pointer of the reading object based on information input by a user in the image based on the image data; a number detection section for recognizing the pointer of the reading object by the mode selected by the mode selection section, so as to respectively detect, based on the image data, a first number positioned on one side surface of the pointer of the reading object and a second number positioned on the other side surface of the pointer of the reading object; and a measurement value detection section for detecting a measurement value indicated by the pointer of the reading object based on the position of the first number, the position of the second number, the position of the pointer, and the respective values of the first number and the second number.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and a computer program.

Background Art

[0002] Recently, services for supporting the meter reading work of meters have been provided. In this service, AI (Artificial Intelligence) is utilized to read the meter values displayed in arithmetic numbers by a device. However, when the meter to be read is an analog meter (numerical display by a needle), since its shape and scale display method are diverse, settings are required for each individual meter display type, the initial settings become complicated, and a work load is generated.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, when it is difficult to recognize the display of the pointer or scale due to the installation environment of the analog meter or the like and meter reading cannot be performed, meter reading could not be carried out. Further, when the shape, color, etc. of the pointer of the analog meter are different from the learned pointer, additional learning had to be performed each time as a response to the new pointer shape and color.

[0005] Embodiments of the present invention have been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus, an information processing method, and a computer program that reduce the load of the meter reading work for reading an analog meter.

Means for Solving the Problems

[0006] The computer program according to the embodiment causes a computer to detect a pointer and recognize the pointer to be read based on image data obtained by photographing an analog meter that indicates a measurement value in a positional relationship between a number and the pointer written on a display panel in a first mode, and to recognize the pointer to be read based on information input by a user in an image based on the image data in a second mode, a mode selection unit that selects either one, and recognize the pointer to be read according to the mode selected by the mode selection unit, and based on the image data, a first number located on one side surface of the pointer to be read and a second number located on the other side surface of the pointer to be read are detected respectively by a number detection unit, and based on the position of the first number, the position of the second number, the position of the pointer to be read, and the respective values of the first number and the second number, it functions as a measurement value detection unit that detects the measurement value indicated by the pointer to be read.

Brief Description of Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 shows a configuration of an information processing system including an information processing apparatus, an information processing method, and a computer program according to an embodiment. In the following description, a case where the information processing apparatus, the information processing method, and the computer program are applied to a needle inspection operation of various meters used in facilities such as factories and offices and various measuring instruments will be described as an example.

[0009] (Configuration) The information processing system shown in FIG. 1 is a system that realizes cloud computing for processing information obtained from meters MT1 to MTn of various facilities and measuring instruments, and includes a portable terminal device 100, a wireless base station 200, a client terminal 300, a server 400, and a network 500.

[0010] Note that the meters MT1 to MTn are assumed to display measured values such as current, voltage, pressure and flow rate of gas and liquid, and values indicating the operating state of facilities and equipment, but may indicate other numerical values. Also, the meters MT1 to MTn include two types: numerical meters and analog meters, and both types may be mixed.

[0011] A numerical meter is an instrument that indicates measured values and the like in either a rotary or digital format. In the case of a digital meter, for example, a configuration can be considered where multiple 7-segment displays showing numbers from 0 to 9 are used to indicate numerical values. In the case of a rotary meter, for example, it is a mechanical type with multiple arithmetic numbers from 0 to 9 written on numbered wheels arranged in a row, and a direct-reading configuration where a multiple-digit numerical value is indicated by the rotation of the numbered wheels can be considered. An analog meter is an instrument in which a pointer moves on a display panel with arithmetic numbers or scales marked as the reading of the meter, and the measured value and the like are indicated by the positional relationship between the numbers or scales and the pointer.

[0012] The portable terminal device 100 is a terminal device carried by a meter reader who checks the meter. It is an information processing device for optically reading the measured values of meters MT1 to MTn installed at the meter-checking site, converting them into data as a form, and transmitting them to the server 400. Details will be described later.

[0013] The wireless base station 200 wirelessly communicates with the portable terminal device 100 and connects the portable terminal device 100 to the network 500. For example, when the network 500 is a LAN constructed within a business office, it is an access point that adopts a wireless LAN (IEEE802.11 series) as the communication method. When the network 500 is a mobile communication network of a telecommunications carrier, it is a base station device that performs wireless communication corresponding to standards such as 3G, 3.9G (such as LTE (registered trademark)), 4G, and 5G.

[0014] That is, wireless LANs and mobile phone networks are just examples, and as long as the portable terminal device 100 can be connected to the network 500 by wireless communication, it is not limited to these methods, and any communication method including Bluetooth (registered trademark) is applicable. Also, the network 500 is not limited to a single network, and a network where multiple networks (for example, an in-house LAN and a mobile communication network) are interconnected may be used.

[0015] The client terminal 300 is installed, for example, in the management department of the users of the needle inspection service provided by the system, and is a terminal used by the operators of that department. It is an information processing device such as a desktop or laptop personal computer, for example.

[0016] Then, the client terminal 300 has the function of downloading the data file uploaded to the server 400 by the portable terminal device 100, and performing operations such as reference / viewing, editing / summarizing, data conversion / file creation, and issuance of forms.

[0017] Also, the client terminal 300 may be equipped with all the functions or some of the data processing functions of the portable terminal device 100 and execute the same processing as the portable terminal device 100. That is, the client terminal 300 may be equipped with the same functions to perform the data processing that can be executed on the portable terminal device 100 instead, except for photographing at the needle inspection site.

[0018] The server 400 forms the center of the cloud computing provided by the system, saves and manages the information uploaded from the portable terminal device 100 and the client terminal 300, performs predetermined data processing on the above information, or provides (downloads) the saved information and the data-processed information to the portable terminal device 100 and the client terminal 300, and realizes data sharing among the users of the system.

[0019] In addition, the server 400 can save the computer resources of each device by undertaking some or all of the data processing by the portable terminal device 100 and the client terminal 300 on behalf of each device. Also, the server 400 provides the OS (Operating System) and application software used by the portable terminal device 100 and the client terminal 300 to each device and updates them with the latest information to realize the service.

[0020] In this way, the database of the server 400 stores the needle inspection data uploaded from the portable terminal device 100, and also stores various programs of the portable terminal device 100 and the client terminal 300.

[0021] The service provided by the server 400 may be provided in a subscription manner to the portable terminal device 100 and the client terminal 300 owned (or leased) by the client receiving the service.

[0022] FIG. 2 is a circuit block diagram schematically showing a configuration example of the portable terminal device shown in FIG. 1. As described above, the portable terminal device 100 is a terminal carried by a needle inspector who performs needle inspection of the meter at the needle inspection site and used as an operator, and is a portable information processing device such as a smartphone, a tablet computer, or a laptop personal computer.

[0023] The portable terminal device 100 has a hardware configuration similar to that of a general computer, and at least includes a communication unit 101, an input unit 102, a display unit 103, a camera 104, a GNSS (Global Navigation Satellite System) receiver unit 105, a storage unit 106, and a control unit 110 connected by a bus.

[0024] The communication unit 101 is a wireless communication interface that establishes a wireless communication link with the wireless base station 200 and communicates, and communicates with the server 400 via the wireless base station 200 and the network 500. The wireless communication corresponds to a communication method according to the specifications of the wireless base station 200. Examples of this communication method include various standards such as wireless LAN (IEEE802.11 series), 3G, 3.9G (such as LTE (registered trademark)), 4G, 5G, Bluetooth (registered trademark), etc., and any of them is acceptable and not limited to these.

[0025] The input unit 102 includes input devices such as a touch panel placed on the display unit 103 described later and key switches provided on the housing of the portable terminal device, and detects input and instructions of various information from the needle inspector. The touch panel can be input using a stylus or a finger, and various methods such as a capacitance method, a resistive film method, and a projection type infrared method can be applied.

[0026] The display unit 103 provides information visually to the needle inspector, and displays, for example, input fields for information, software keys, and various images (photos, CG (computer graphics) images).

[0027] The input fields for information and software keys are displayed in correspondence with operations on the aforementioned input unit 102 under the control of the control unit 110 described later. As devices used for display, various display devices such as a liquid crystal panel, an organic EL (Electro Luminescence) panel, and electronic paper can be applied.

[0028] The camera 104 is a digital camera including an optical system having a plurality of lenses and a zoom mechanism for varying the distance between these lenses, an imaging unit having an image sensor such as CMOS (Complementary MOS), and a signal processing unit for generating image data in a predetermined format (for example, JPEG (Joint Photographic Experts Group)) from the imaging signal obtained by the imaging unit.

[0029] Note that additional information (for example, Exif (Exchangeable image file format) data) indicating the shooting date and time, location, shooting conditions (shutter speed, aperture value, angle of view, latitude and longitude, etc.) is added and recorded to the image data obtained by the camera 104 by the control unit 110.

[0030] The GNSS receiving unit 105 performs position measurement (positioning) using signals transmitted from a plurality of GNSS satellites, ground reference stations, etc., and receives the current time, etc.

[0031] The storage unit 106 stores the OS and application software of the control unit 110 described later, data generated during the operation of the application software, various parameters, data input from the meter reader, image data captured by the camera 104 (such as an image capturing the measured values of the meters MT1 to MTn), data acquired (downloaded) from the server 400, and other temporary data for information processing. Recording devices such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory such as an SSD (Solid State Drive), and an HDD (Hard Disk Drive) are provided in combination according to the characteristics of the data. The storage unit 106 also stores various data such as form data 106a, setting information data 106b, and meter image data 106c.

[0032] The form data 106a is data created to aggregate the measured values of each meter for each predetermined meter reading period. When new data is created, it is uploaded to the server 400. Specifically, the form data 106a is data created for each meter reading site. The form data 106a is data in which information such as "meter information", "meter image", "current measured value", "difference from the previous time", "previous measured value", and "difference obtained previously" is associated. When multiple meters are installed at the meter reading site, the above information is included for each meter.

[0033] Here, the "meter information" includes a master code for identifying the meter, identification information indicating the object measured by the meter (for example, tenant, parking lot, shared space, etc.), and the type (for example, gas, electricity, water, etc.). The "meter image" is an image serving as the basis for the measured value, that is, an image captured during the meter reading.

[0034] The "current needle reading value" is the value read in the current needle reading (the latest needle reading value). The "difference from the previous time" is the difference between the current needle reading value and the needle reading value of the previous time (one time before the latest time). The "previous needle reading value" is the value read in the previous needle reading. The "difference obtained in the previous time" is the difference between the needle reading value of the previous time and the needle reading value of the time before the previous time (two times before the latest time).

[0035] The setting information data 106b is setting information necessary for totaling the needle reading values of each meter. Specifically, similar to the form data 106a, it is created for each meter at each needle reading site. Further, the setting information data 106b includes setting information of a standard mode for performing the needle reading process of the meter. In the present embodiment, as a mode for recognizing the pointer to be read during the needle reading process, an auto mode (first mode) that automatically recognizes the pointer to be read of the analog meter based on the image data obtained by photographing the analog meter, and a manual mode (second mode) that recognizes the pointer to be read based on the information input by the user in the image based on the image data obtained by photographing the analog meter, either one of them is stored in the setting information data 106b as the standard mode. For example, the standard mode for reading the needle may be that the auto mode is initially set in the setting information data 106b and can be switched to the manual mode by the operation of the needle reader (user). For example, a display control unit 1,13 described later controls the display unit 103 to display a graphical user interface (hereinafter, GUI (Graphical User Interface)) in which the user can input the needle reading mode by operating the input unit 102.

[0036] The meter image data 106c is image data obtained by the camera 104 photographing the meter for needle reading. When it is actually used to obtain the needle reading value, the necessary part is cut out from the image and recorded in the form data 106a as the meter image.

[0037] The control unit 110 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a chipset, a RAM, a ROM, etc., and comprehensively controls each part of the portable terminal device 100.

[0038] The ROM stores firmware and setting values (various parameters). The CPU reads the OS and application software from the storage unit 106 into the RAM according to the above firmware, executes the OS and application software, and uses the RAM as a work area (working area) to realize various control functions.

[0039] The control unit 110 realizes at least the following control functions by executing the OS and application software. That is, the control unit 110 functions as a communication control unit 111, an input control unit 112, a display control unit 113, an image processing unit 114, a position information processing unit 115, and a stitch data aggregation processing unit 116 by executing software (program) on the processor, and these functions can cooperate with each other.

[0040] Note that it is also possible to integrate some or all of these functions with other functions, or to divide the functions of the communication control unit 111, the input control unit 112, the display control unit 113, the image processing unit 114, the position information processing unit 115, and the stitch data aggregation processing unit 116 into multiple function blocks from another perspective and explain them in another expression.

[0041] The communication control unit 111 controls the communication unit 101 to establish a wireless communication link with the wireless base station 200 according to a predetermined communication protocol, and further communicates with the server 400 through the network 500. For example, it downloads data from the server 400 and uploads data to the server 400.

[0042] The input control unit 112 controls the input unit 102, interprets the inspector's request from the operation of the inspector on the input unit 102 from the result detected by the input unit 102, accepts the input of information (such as a character string), the import and association change of data, the import of image data captured by the camera 104, etc.

[0043] The display control unit 113 controls the display unit 103 to display various information including characters and images, and a graphical user interface (hereinafter referred to as GUI (Graphical User Interface)). Examples of the GUI are the input fields and software keys for the above-described information, and these displays correspond to the input through the touch panel of the input unit 102. In addition, a guide display indicating the image capture position is performed.

[0044] The image processing unit 114 performs image processing on the image data captured by the camera 104, and includes, for example, a function as a recognition engine and a function as an analysis engine.

[0045] The recognition engine performs a process of recognizing various information from an image based on the image data obtained by the camera 104 by pattern recognition using, for example, AI (Artificial Intelligence). Specifically, the recognition engine, for example, detects a QR code (registered trademark), distinguishes (determines) whether the type of the meter shown in the image is numerical data or analog data, distinguishes (determines) whether the meter is of a type including a plurality of pointers or a type including one pointer, detects the meter portion that displays the measured value on the display panel of the meter, and also has a function of detecting an object (a numerical string of the measured value of the numerical data, a pointer of the analog meter, a number written on the display panel, etc.) and its arrangement (coordinates) in the meter portion.

[0046] On the other hand, the analysis engine performs a process of analyzing the information detected by the recognition engine. Specifically, for the image of the meter part, the analysis engine decodes codes such as QR codes (two-dimensional codes) and barcodes (one-dimensional codes), or converts the measured values of numerical data into text through character recognition processing such as optical character recognition (OCR), or detects the value indicated by the pointer from the positional relationship of the objects marked on the meter part of the analog meter.

[0047] Note that multiple types of character recognition processing are prepared for the analysis engine. Each analysis engine corresponds to each type of meter, and uses an analysis algorithm suitable for the display (numerical sequence) of the measured values of the corresponding meter to convert the displayed measured values into text.

[0048] Specifically, since the fonts of arithmetic numbers and the combinations of numbers and background colors vary depending on the meter, analysis engines with analysis algorithms suitable for the display of each meter are prepared.

[0049] The position information processing unit 115 is equipped with an acceleration sensor, a gyro sensor, etc. Based on the detection results of these, the positioning result by the GNSS receiving unit 105, and / or the positioning information obtained from the radio base station 200 by the communication control unit 111, the position of the portable terminal device 100 is positioned.

[0050] The needle inspection data aggregation processing unit 116 comprehensively controls various processes related to the aggregation of needle inspection data, and includes a numerical data needle inspection unit 116a, an analog meter needle inspection unit 116b, an invoice creation unit 116c, and a correction processing unit 116d. The numerical data needle inspection unit 116a uses the image processing unit 114 as needed to perform the process of reading the numerical sequence of the measured values displayed in the numerical data as the needle inspection value.

[0051] The analog meter needle detection unit 116b uses the image processing unit 114 as necessary to perform a process of reading the value indicated by the needle on the analog meter as the measured value, and is an example of a mode selection unit, a needle detection unit, a digit detection unit, a measured value detection unit, a straight line creation unit, an arc creation unit, an intersection detection unit, an image data generation unit, a recording unit, a meter reading completion determination unit, and a needle reading completion determination unit.

[0052] The mode selection unit has a function of selecting either an auto mode (first mode) for detecting the needle and recognizing the needle to be read based on the image data obtained by photographing the analog meter that indicates the measured value by the positional relationship between the digits and the needle written on the display panel, or a manual mode (second mode) for recognizing the needle to be read based on the information input by the user in the image based on the image data.

[0053] The digit detection unit has a function of recognizing the needle to be read according to the mode selected by the mode selection unit and detecting, based on the image data, a first digit located on one side surface of the needle to be read and a second digit located on the other side surface of the needle to be read, respectively. The measurement detection unit has a function of detecting the measured value indicated by the needle to be read based on the positions of the first digit, the position of the second digit, the position of the needle to be read, and the respective values of the first digit and the second digit.

[0054] The straight line creation unit and the arc creation unit have a function of creating a straight line and an arc on the image obtained by photographing the analog meter by the image processing accompanying the needle detection process. The intersection detection unit has a function of detecting the intersections of the straight lines and arcs created on the image obtained by photographing the analog meter by the image processing accompanying the needle detection process. The image data generation unit has a function of performing trimming (cropping) etc. of the image using the image data photographed by the camera and generating image data suitable for performing the needle detection process. The recording unit has a function of recording the calculation results in the needle detection process, the image data used in the needle detection process, etc. in the storage unit 106.

[0055] Based on the input information on whether the reading of the pointer values of all (or necessary) pointers of the analog meter has been completed, the pointer reading completion determination unit enables the detection of the pointer values of other pointers of the analog meter as necessary. Based on the input information on whether the needle inspection process of all (or necessary) analog meters has been completed, the meter reading completion determination unit enables the start of the needle inspection process of other analog meters as necessary.

[0056] The form creation unit 116c performs a process of creating form data based on the needle inspection values read by the numerical data needle inspection unit 116a and the analog meter needle inspection unit 116b.

[0057] The correction processing unit 116d receives an instruction to correct the needle inspection value according to an instruction from the needle inspector, and performs a process of correcting the needle inspection value read by the numerical data needle inspection unit 116a and the analog meter needle inspection unit 116b.

[0058] (Operation) Next, the operation of the information processing system having the above configuration will be described. In the following description, among the processes in the portable terminal device 100, in particular, the aggregation process of the needle inspection data (hereinafter referred to as the needle inspection data aggregation process) will be described.

[0059] This needle inspection data aggregation process is performed by the needle inspection data aggregation unit 116, supports the needle inspection work on the meters MT1 to MTn by the needle inspector, reads the measured values displayed on the meters MT1 to MTn, and creates aggregated form data 106a.

[0060] Also, this needle inspection data aggregation process is realized by the control unit 110 that executes application software (program) functioning as the needle inspection data aggregation unit 116. In the following description, it will be described as one application software, but it may also be realized by a plurality of application softwares operating in cooperation or continuously.

[0061] When the application software is executed, the form creation unit 116c of the needle inspection data aggregation processing unit 116 displays the template of the form data 106a. Specifically, the form creation unit 116c reads out the template data of the form data 106a stored in the storage unit 106, and outputs the read template data to the display control unit 113. In contrast, the display control unit 113 displays the form template on the display unit 103 based on the above template data.

[0062] Subsequently, the needle inspection data aggregation processing unit 116 gives an instruction to the display control unit 113 to display a request for input of the type of meter to be needle-inspected on the display unit 103. That is, a display is made to ask the needle inspector whether to perform needle inspection on numerical data or analog data, and to allow selection, and the input control unit 112 receives the selection of the needle inspector through the input unit 102.

[0063] Here, when the needle inspector selects needle inspection of numerical data, the input control unit 112 detects this input, and in response to this detection result, the needle inspection data aggregation processing unit 116 starts needle inspection by the numerical data needle inspection unit 116a. That is, the numerical value displayed in the numerical data is read by OCR and converted into text, and then the form creation unit 116c inputs the text-converted needle inspection value into the form data 106a.

[0064] On the other hand, when the needle inspector selects needle inspection of analog data, the input control unit 112 detects this input, and in response to this detection result, the needle inspection data aggregation processing unit 116 performs needle inspection by the analog data needle inspection unit 116b. Hereinafter, the details will be described with reference to FIGS. 3 to 7.

[0065] As described above, the needle inspector may instruct whether to perform needle inspection on numerical data or analog data through the input unit 102. Alternatively, as described above, the camera 104 may photograph the meter to be needle-inspected, and the recognition engine may distinguish (determine) whether the type of meter shown in the image is numerical data or analog data.

[0066] FIG. 3 to FIG. 5 are flowcharts for explaining an example of the needle inspection data aggregation process for needle inspection of an analog meter by the information processing apparatus according to an embodiment. FIG. 6 to FIG. 8 are diagrams schematically showing an example of a display when performing needle inspection of an analog meter in the information processing apparatus according to an embodiment.

[0067] As described above, this needle inspection data aggregation process is realized by the control unit 110 that executes the application software (program) functioning as the needle inspection data aggregation unit 116 (analog meter needle inspection unit 116b).

[0068] In step S1 of FIG. 3, the analog meter needle inspection unit 116c of the mobile terminal 100 reads the setting information data 106b from the storage unit 106 and proceeds to step S2. The read setting information data 106b is used as an initial setting parameter (initial setting value) in subsequent processing.

[0069] As an example of the setting information data 106b, for various analog meters, information such as the minimum and maximum values of the numerical values displayed on the display panel associated with the meter identification information, or learning data obtained by AI (Artificial Intelligence) learning for identifying the meter is included.

[0070] The learning data is generated by machine learning based on information such as the center point of the meter, the start and end points of the movable range of the pointer, the bottom point at the bottom of the display panel, the range of the pointer (minimum value, maximum value), and threshold values set in the image of the analog meter.

[0071] In step S2, the analog meter needle inspection unit 116c activates the camera 104, images the analog meter MT (for example, any one of the meters MT1 to MTn) at the shooting composition (angle) by the needle inspector to acquire image data, and the processing procedure proceeds to step S3.

[0072] Specifically, in step S2, the camera 104 is activated, and the video captured by the camera 104 is displayed on the display unit 103. The needle inspector checks this display, adjusts the shooting composition, and performs a shutter operation in a state where the entire analog meter MT is captured. At this time, the needle inspector may visually confirm the composition for shooting the entire analog meter MT and press the shooting button B1 displayed on the display unit 103 to perform a shutter operation, or the shutter operation may be automatically performed in response to the entire analog meter MT being recognized by the image processing of the image processing unit 114. Thereby, image data in which the entire analog meter MT is captured is obtained, and on the display unit 103, for example, as shown in FIGS. 6 to 8, the captured analog meter MT is displayed. In this display, the display control unit 113 displays a GUI (Graphical User Interface) for needle inspection on the display unit 103 according to an instruction from the analog meter needle inspection unit 116b.

[0073] In the GUI for needle inspection, for example, a shooting guide frame (not shown) for shooting the meter is displayed, and the video captured by the camera 104 is displayed as it is within the shooting guide frame. Outside the shooting guide frame, for example, a gray mask display with a reduced transmittance is performed to prompt the operating needle inspector to place the analog meter MT within the shooting guide frame. Seeing this, the needle inspector moves the camera 104 of the portable terminal device 100 back and forth, left and right, and up and down while holding it over the analog meter MT to change the shooting composition and adjust it so that the analog meter MT fits within the shooting guide frame.

[0074] In the examples of the GUI shown in FIGS. 6 and 8, the zoom key Z and the software keys A1 to A4 are displayed when shooting the analog meter. The zoom key Z instructs the image processing unit 114 to perform a process of enlarging and displaying the image data captured by the camera 104.

[0075] For example, "1X" corresponds to no magnification process, "2X" corresponds to a process of doubling the viewing angle for display compared to "1X", "3X" corresponds to a process of tripling the viewing angle for display compared to "1X", and "maximum" corresponds to a process of maximizing the viewing angle for display according to the performance compared to "1X". The image processing unit 114 performs a process corresponding to the operated zoom key Z and displays the image captured by the camera 104.

[0076] In the above, the digital zoom that changes the viewing angle by the image processing of the image processing unit 114 has been described as an example. However, the optical zoom mechanism provided in the camera 104 may be driven and controlled to change the viewing angle according to the operated software key, or alternatively, a combination of digital zoom by the image processing unit 114 and optical viewing angle change control may be employed.

[0077] "Return to form" of the software key A1 is for switching from the meter reading process screen to the screen for displaying the form. "QR" of the software key A2 is for instructing, for example, the reading of a QR code. "Reshoot" of the software key A3 is for instructing the reshooting of the meter. "Registration" of the software key A4 is for instructing the registration of the result of the meter reading in the form.

[0078] In step S3, the analog meter reading unit 116c displays the photographed analog meter MT on the display unit 103 based on the image data obtained in step S2, and detects the notation numbers on the display panel of the analog meter MT shown in the image data. Note that the recognition engine of the image processing unit 114 is used for this detection.

[0079] Also in step S3, the analog meter reading unit 116c surrounds each of the detected notation numbers with a rectangular frame line (numerical frame) on the displayed analog meter MT and proceeds to step S4.

[0080] FIG. 7 shows an example of the frame lines surrounding the notation numerals and the pointer, respectively, displayed on the image of the analog meter MT. These displays are performed through the display control unit 113, and the same applies to the displays described below. Note that the detection of the notation numerals and the pointer on the analog meter MT can be realized, for example, by using AI.

[0081] That is, based on the learning data and the image data included in the setting information data 106b, the analog meter needle detection unit 116c identifies the type of the analog meter MT and further detects the notation numerals and the pointer on the display panel from the learning data for the identified analog meter MT.

[0082] In step S4, the analog meter needle detection unit 116c detects the position (coordinates) of the center of the frame line surrounding the notation numeral displayed in step S3 and proceeds to step S5. The coordinates detected here are referred to as notation numeral coordinates in the following description. The notation numeral coordinates are based on the two-dimensional coordinate axes set on the displayed analog meter MT, and the coordinates in the following description are also based on the above coordinate axes.

[0083] In step S5, the analog meter needle detection unit 116c obtains an approximated circle (or pseudo-circle) passing through the center of the frame line based on the notation numeral coordinates detected in step S4. That is, the analog meter needle detection unit 116c obtains a circle on which the notation numeral coordinates exist on the circumference. Furthermore, in step S6, the analog meter needle detection unit 116c calculates the coordinates of the center point O of the circle obtained in step S4 and its radius R and proceeds to step S7. Note that in the display example of the image of the analog meter MT shown in FIG. 7, the notation numeral coordinates, the circle (or pseudo-circle, arc), and the center point O of the pseudo-circle are displayed at their corresponding positions on the image of the analog meter MT.

[0084] Subsequently, the analog meter needle detection unit 116b accesses the setting information data 106b in the storage unit 106 to determine whether the standard setting of the needle reading mode of the analog meter MT is the auto mode (step S7).

[0085] When the standard setting of the needle reading mode is the auto mode (step S7, YES), for example, at the instruction from the analog meter needle detection unit 116b, the display control unit 113 displays on the display unit 103 a mode switching button for the needle inspector (user) to select switching from the auto mode to the manual mode, and the analog meter needle detection unit 116b waits for the content input by the needle inspector to be transmitted from the input control unit 112 through the input unit 102.

[0086] When the input control unit 112 detects the input information of the needle inspector and it is transmitted from the input control unit 112 that the mode switching button has been pressed (step S8, YES), the analog meter needle detection unit 116b performs the needle reading process in the manual mode.

[0087] When the input control unit 112 detects the input information of the needle inspector and it is transmitted from the input control unit 112 that the mode switching button has not been pressed (for example, the standard setting mode has been selected) (step S8, NO), the analog meter needle detection unit 116b performs the needle reading process in the auto mode.

[0088] When the standard setting of the needle reading mode is not the auto mode (step S6, NO), for example, at the instruction from the analog meter needle detection unit 116b, the display control unit 113 displays on the display unit 103 a mode switching button for the needle inspector (user) to select switching from the manual mode to the auto mode, and the analog meter needle detection unit 116b waits for the content input by the needle inspector to be transmitted from the input control unit 112 through the input unit 102.

[0089] When the input control unit 112 detects the input information of the needle inspector and it is transmitted that the mode switching button has been pressed from the input control unit 112 (step S12, YES), the analog meter needle inspection unit 116b performs the needle reading process in the auto mode.

[0090] When the input control unit 112 detects the input information of the needle inspector and it is transmitted that the mode switching button has not been pressed (for example, the standard setting mode has been selected) from the input control unit 112 (step S12, NO), the analog meter needle inspection unit 116b performs the needle reading process in the manual mode.

[0091] In the needle reading process, the operations of recognizing the pointer to be read are different between the auto mode and the manual mode.

[0092] FIG. 7 schematically shows an example of the operations of the processes of reading the needle in the auto mode and the manual mode in the needle inspection process of the analog meter by the information processing apparatus according to an embodiment. The upper part of FIG. 7 is a diagram for explaining an example of the operation of the process of reading the needle in the auto mode. The lower part of FIG. 7 is a diagram for explaining an example of the operation of the process of reading the needle in the manual mode. The analog meter MT to be detected in the present embodiment includes a plurality of pointers. Here, as an example, the case where the analog meter MT having a black needle and a red needle is taken as the detection target will be described. Note that the number of pointers of the analog meter MT is not limited to two, and it is also possible to read the needles of the analog meter MT having three or more pointers in the information processing apparatus of the present embodiment.

[0093] Subsequently, the analog meter needle inspection unit 116b uses the recognition engine of the image processing unit 114 to detect the display panel-like object (pointer) of the analog meter MT shown in the image and its coordinates from the imaging data obtained in step S2 (step S9).

[0094] In this embodiment, the analog meter needle detection unit 116b first detects the frame Sq1 of the black needle (first needle N1). Specifically, if the image obtained in step S2 is as shown in FIG. 7, for example, the analog meter needle detection unit 116b uses the recognition engine of the image processing unit 114 to detect a rectangular frame Sq1 surrounding the first needle N1 of the analog meter MT. The rectangular frame Sq1 detected here includes, for example, sides extending in the horizontal direction and sides extending in the vertical direction of the image of the analog meter MT, and is a rectangular frame Sq1 circumscribing the needle.

[0095] Subsequently, in the same manner as the detection of the frame Sq1 of the black needle (first needle N1), the analog meter needle detection unit 116b detects a rectangular frame Sq2 surrounding the red needle (second needle N2). That is, the analog meter needle detection unit 116b uses the recognition engine of the image processing unit 114 to detect the first needle N1 and the second needle N2 of the analog meter MT as shown in FIG. 7, sets a virtual X-Y coordinate plane on the image of the display panel, and detects the coordinates of both ends (starting point and ending point) of the first needle N1 and the second needle N2. Note that the order of detecting a plurality of needles is not limited to the above.

[0096] Note that various methods can be considered for the method of recognizing the numerical frame and the needle by the recognition engine. The recognition engine can use, for example, images of various analog meters and numerical images as input data, and use the information of the numerical frame as teacher data to generate a learned model that can discriminate and recognize the numerical frame and the numerical value through machine learning, and then recognize the numerical frame and the numerical value. Through machine learning, for example, in the image of an analog meter, it is possible to let the computer learn that the numerical values are arranged side by side on an arc, the magnitude relationship between the numbers inside and outside the numerical frame, the magnitude relationship and positional relationship between a plurality of numerical values included in the numerical frame, etc. By recognizing the numerical frame using the above-mentioned learned model, it is no longer necessary to preset the arrangement and size of the numerical values for each type of analog meter, and it becomes possible to automatically set the conditions for recognizing the numerical values of various analog meters. As a result, the recognition engine can identify the numerical values arranged on the left and right of any numerical value and automatically detect negative values.

[0097] Further, the recognition engine may recognize, for example, an object extending outward from near the center of the image (i.e., the center of the analog meter MT) and having a length equal to or greater than a certain ratio with respect to the image size as the first pointer N1 or the second pointer N2, or recognize an object having a specific color (red or black) and an aspect ratio equal to or greater than a predetermined value as the first pointer N1 or the second pointer N2, or recognize an object having a specific shape as the first pointer N1 or the second pointer N2. Further, the recognition engine may use a learned model generated to be able to discriminate and recognize the first pointer N1 and the second pointer N2 by machine learning using an image including at least one of the first pointer N1 and the second pointer N2 to recognize the first pointer N1 and the second pointer N2. In addition, those skilled in the art will easily understand that other well-known methods can be appropriately used to achieve this.

[0098] Subsequently, in accordance with an instruction from the analog meter needle detection unit 116b, the display control unit 113 displays on the display unit 103 a display prompting the needle inspector to select a needle (step S10). Here, when a plurality of needles N1 and N2 are detected from the image of the analog meter MT, the display control unit 113 may display a plurality of images of the analog meter MT on the display unit 103, and display on the display unit 103 an image (such as a frame line) corresponding to a different needle on each of the plurality of images of the analog meter MT. The needle inspector can select a needle by pressing an image on which an image corresponding to the needle to be read is displayed from among the plurality of images of the analog meter MT on the display unit 103.

[0099] The analog meter needle detection unit 116b waits for the content input by the needle inspector to be transmitted from the input control unit 112 through the input unit 102. When the input control unit 112 detects the input information of the needle inspector and, for example, it is transmitted that a region of any needle has been pressed (any needle has been selected) from the input control unit 112 to the analog meter needle detection unit 116b, the analog meter needle detection unit 116b detects the tip of the needle selected by the needle inspector (step S11).

[0100] In the rectangular frame surrounding the pointer selected by the needle inspector, the analog meter needle detection unit 116b detects, as the coordinates of the tip of the pointer, the coordinates of the corner of the rectangle that is farther from the center O of the circle obtained in step S5 among the two corners near both ends of the pointer. For example, when the first pointer N1 is selected by the needle inspector, the analog meter needle detection unit 116b detects, in the rectangular frame Sq1 surrounding the first pointer N1, the coordinates of the corner P1 that is farther from the center O among the two corners near both ends of the first pointer N1 as the coordinates of the tip of the pointer. As described above, the analog meter needle detection unit 116b performs the recognition process of the pointer in the auto mode.

[0101] When performing the recognition process of the pointer in the manual mode, the analog meter needle detection unit 116b uses the recognition engine of the image processing unit 114 to detect an object (analog meter MT) shown in the image from the captured data obtained in step S2, and cuts out the detected object (analog meter MT) (step S13).

[0102] Subsequently, at the instruction of the analog meter needle detection unit 116b, the display control unit 113 performs a display prompting the needle inspector to drag and surround the area of the analog meter MT including the pointer to be read in the image cut out in step S13 on the display unit 103. The analog meter needle detection unit 116b waits for the content input by the needle inspector to be transmitted from the input control unit 112 through the input unit 102.

[0103] When the input control unit 112 detects the input information of the needle inspector and it is transmitted from the input control unit 112 to the analog meter needle detection unit 116b that an input has been made to surround the area of the display image, the analog meter needle detection unit 116b instructs the display control unit 113 to cut out and display the image of the area selected by the needle inspector by dragging, and to display a prompt for the needle inspector to drag a line on the pointer to be read. The analog meter needle detection unit 116b waits for the content input by the needle inspector to be transmitted from the input control unit 112 through the input unit 102.

[0104] When the analog meter needle detection unit 116b detects the input information of the needle inspector by the input control unit 112 and is notified that an input to draw a line on the display image has been made from the input control unit 112 (step S14), it uses the recognition engine of the image processing unit 114 to set the end of the input line that is farther (shorter) from the center point O calculated in step S6 as the tip P1 of the pointer (step S15). Here, when the line input by the needle inspector is not a straight line, the analog meter needle detection unit 116b corrects the input line to a straight line by the recognition engine, calculates the distances between the coordinates of each of the two ends of the corrected straight line and the coordinates of the center of the numerical value frame, and can set the end with the shorter distance from the center of the numerical value frame as the tip of the pointer.

[0105] Specifically, the analog meter needle detection unit 116b instructs the display control unit 113 to display the straight line input by the needle inspector on the image of the analog meter MT on the display unit 103. The analog meter needle detection unit 116b uses the recognition engine of the image processing unit 114 to detect both ends of the straight line input by the needle inspector and acquires the coordinates of each of the detected ends.

[0106] The analog meter needle detection unit 116b calculates the distance d1 between one end and the center point O and the distance d2 between the other end and the center point O using the coordinates of the detected both ends and the center point O, compares the distance d1 and the distance d2, and sets the end P1 that is farther (shorter) from the center point O as the tip of the pointer to be read.

[0107] By setting the tip and the fulcrum of the pointer to be read as described above, the analog meter needle detection unit 116b can recognize the position of the pointer manually input by the needle inspector. The process of reading the numerical value indicated by the pointer after recognizing the pointer for numerical value reading in either the auto mode or the manual mode is common to both the auto mode and the manual mode.

[0108] The analog meter detection unit 116b sets the position of the pointer as the straight line connecting the tip P1 of the pointer and the center point O of the circle (step S16). Further, the analog meter detection unit 116b creates a linear function Y = c*X + d that connects the selected tip P1 of the pointer and the center point O (calculates the values of c and d), and further, in order to indicate that the selected pointer has been detected, based on the coordinates of the tip P1 of the pointer and the coordinates of the center point O, it instructs the display control unit 113 to display a line segment that overlaps in the longitudinal direction with the pointer (a part of the straight line passing through the tip P1 of the pointer and the center point O).

[0109] Subsequently, the analog meter detection unit 116c arranges the notation digit coordinates detected in step S4 in order (step S17). Specifically, the analog meter detection unit 116c specifically arranges the notation digit coordinates on the circumference in a clockwise order starting from a point on the circumference below the center point O of the circle. This is in consideration of the fact that the reading value of a general analog meter changes (increases) in a clockwise direction.

[0110] Subsequently, the analog meter detection unit 116c reads the numerical value of the notation digit within the frame line surrounded in step S3 (step S18). The reading (detection) of the numerical value of the notation digit can be realized, for example, by using AI. That is, based on the learning data and image data included in the setting information data 106b, the analog meter detection unit 116c can identify the type of the analog meter MT, etc. Furthermore, the numerical value of the notation digit on the display panel can be detected from the learning data for this identified analog meter MT.

[0111] Next, the analog meter detection unit 116c determines whether there is a "0" in the numerical value read in step S18 (step S19). Here, if there is a "0" in the numerical value read in step S18, the processing procedure proceeds to step S20, and if there is no "0", the processing procedure proceeds to step S23.

[0112] In step S20, based on the array performed in step S17 and the numerical value read in step S18, the analog meter needle detection unit 116c determines whether there is a numerical value on the left side of the numerical digit of "0" when viewed from the center point O. Here, if there is a numerical value on the left side, the processing procedure proceeds to step S21, and if there is no numerical value on the left side, the processing procedure proceeds to step S24.

[0113] In step S21, for the numerical values among the numerical values read in step S18 that exist on the left side of the numerical value "0", the analog meter needle detection unit 116c converts them into negative values of the same magnitude and proceeds to step S24. For the numerical values that exist on the right side, they remain as positive values.

[0114] For example, in the case of an analog meter MT where numerical values "0.05" and "0.1" exist on the left side of the numerical value "0", the analog meter needle detection unit 116c converts these numerical values into "-0.05" and "-0.1" respectively.

[0115] In step S22, based on the numerical value frame arranged in step S17 and the numerical value read in step S18, the analog meter needle detection unit 116c compares two adjacent numerical values in the arranging direction and determines whether there is a numerical value that is greater than or equal to the numerical value on the right side. That is, it determines whether there is a numerical value where the numerical value on the left side is greater than or equal to the numerical value on the right side when viewed from the center point O. If there is a numerical value that is greater than or equal to the numerical value on the right side, the processing procedure proceeds to step S23, and if there is no numerical value that is greater than or equal to the numerical value on the right side, the processing procedure proceeds to step S24.

[0116] In step S23, for the numerical values among the numerical values read in step S18 that are greater than or equal to the numerical value on the right side, the analog meter needle detection unit 116c reverses the sign of these numerical values and the processing procedure proceeds to step S24.

[0117] For example, in the case of the analog meter MT where the numerical value "0.05" exists on the right side of the numerical value "0.1" at the left end, the analog meter needle inspection unit 116c converts the numerical value "0.1" at the left end to "-0.1". Similarly, in the case of the analog meter MT where the numerical value "0" exists on the right side of the numerical value "0.05" on the left side, the analog meter needle inspection unit 116c converts the numerical value "0.05" to "-0.05".

[0118] In step S24, the analog meter needle inspection unit 116c detects the coordinates of the intersection point between the line segment connecting the coordinates of the tip P1 of the pointer detected in either step S15 or step S16 and the coordinates of the center point O of the circle detected in step S5, and the circumference of the circle obtained in step S5. Then, regardless of the presence or absence of the intersection point, the processing procedure proceeds to step S25.

[0119] In step S25, the analog meter needle inspection unit 116c determines whether the coordinates of the intersection point could be detected in step S24. If the coordinates of the intersection point could be detected (i.e., there is an intersection point), the processing procedure proceeds to step S27. If the coordinates of the intersection point cannot be detected, such as when there is no intersection point, the processing procedure proceeds to step S26.

[0120] In step S26, the analog meter needle inspection unit 116c extends the line segment connecting the tip P1 of the pointer and the center point O, creates an intersection point between the extended line segment and the circumference, and detects the coordinates of that intersection point. Then, the processing procedure proceeds to step S27. The detection of the coordinates of the intersection point can be realized, for example, by extending the line segment in the direction from the center point O to the tip P1 based on the coordinates of the tip P1 of the pointer and the coordinates of the center point O.

[0121] In step S27, the analog meter needle inspection unit 116c searches for the notation numbers on the circumferences on the left and right of the coordinates of the intersection point detected in step S24 or step S26, and determines whether those notation numbers exist.

[0122] In the example of FIG. 7, on the circumference to the left of the intersection coordinates of the first pointer N1 and the circumference (located on one side surface of the pointer), the notation number "2" (the first number) exists, and on the circumference to the right (located on the other side surface of the pointer), the notation number "3" (the second number) exists.

[0123] Here, when notation numbers can be detected on both the left and right, the processing procedure proceeds to step S28. When the notation number exists only on one of the left or right, as the pointer value, the minimum value (when only the notation value on the right side of the intersection is detected) or the maximum value (when only the notation value on the left side of the intersection is detected) is detected, and the processing procedure proceeds to step S33.

[0124] In the example of FIG. 7, for example, on the left circumference of the intersection coordinates of the first pointer N1 and the circumference, the notation number "2" exists, and on the right circumference, the notation number "3" exists. In this case, the processing procedure proceeds to step S28.

[0125] On the other hand, for example, when there is no notation number on the left side (up to the starting point of the circumference) of the intersection of the pointer and the circumference as seen from the center point O, as the pointer value, the minimum value based on the initial setting value is detected and determined, and the processing procedure proceeds to step S33.

[0126] Also, for example, when there is no notation number on the right side (up to the end point of the circumference) of the intersection of the pointer and the circumference as seen from the center point O, as the pointer value, the maximum value based on the initial setting value is detected and determined, and the processing procedure proceeds to step S33.

[0127] Also, for example, when the pointer overlaps with a notation number (e.g., "4") and the notation number "4" is not detected, the notation value "4" is not used in the determination of step S27. However, since the other left and right notation numbers (e.g., "3" and "5") are detected respectively, after the existence of these notation numbers is detected in step S27, the processing procedure proceeds to step S28.

[0128] Furthermore, for example, when the pointer overlaps the indicated number "4" and for some reason the indicated number "5" is not detected, the indicated numbers "4" and "5" will not be used in the determination of step S27. However, since the other left and right indicated numbers "3" and "6" are detected, after these indicated numbers are detected as the indicated number on the right side and the indicated number on the left side of the pointer respectively, the processing procedure proceeds to step S28.

[0129] In step S28, the analog meter detecting unit 116c first detects the angle (first angle) formed by the line segment connecting the coordinates of the indicated number ("2") on the left side of the intersection point of the pointer and the circumference and the coordinates of the center point O, and the line segment (first pointer) connecting the coordinates of the intersection point and the coordinates of the center point O, among the two indicated numbers detected in step S27.

[0130] Next, the analog meter detecting unit 116c detects the angle (second angle) formed by the line segment connecting the coordinates of the indicated number ("3") on the right side of the intersection point of the pointer and the circumference and the coordinates of the center point O, and the line segment (first pointer) connecting the coordinates of the intersection point and the coordinates of the center point O, among the two indicated numbers detected in step S27, and proceeds to step S29.

[0131] In step S29, the analog meter detecting unit 116c calculates the pointer value corresponding to the intersection point of the pointer and the circumference based on the first angle S1, the second angle S2, the radius R of the circumference, the numerical value VL of the left indicated number, and the numerical value VR of the right indicated number. After this, the processing procedure proceeds to step S30.

[0132] For example, the analog meter detecting unit 116 can calculate the pointer value VD as follows based on the ratio of each distance (angle) to the sum of the distance (corresponding to the first angle S1) between the position of the first number and the position of the pointer to be read and the distance (corresponding to the second angle S2) between the position of the second number and the position of the pointer, the value VL of the first number, and the value VR of the second number. VD=(S1 / (S1+S2))*VR+(S2 / (S1+S2))*VL =(S1*VR+S2*VL) / (S1+S2)

[0133] In step S30, the analog meter needle detection unit 116c detects the maximum number of digits among the digits after the decimal point based on the numerical value of the notation number read in step S18, and the processing procedure proceeds to step S31.

[0134] For an analog meter as shown in FIGS. 6 to 8, the read numerical value includes the digits after the decimal point, and the analog meter needle detection unit 116c detects, for example, that the maximum number of digits after the decimal point is 1. For example, for an analog meter with notation numbers "0.1" and "0.05", the analog meter needle detection unit 116c detects that the maximum number of digits is 2 based on "0.05".

[0135] In step S31, the analog meter needle detection unit 116c obtains the number of significant digits after the decimal point as the value obtained by adding 1 to the maximum number of digits detected in step S30. Then, the analog meter needle detection unit 116c determines the value up to the number of significant digits after the decimal point among the calculation results of step S29 as the pointer value, and the processing procedure proceeds to step S32. For example, in the case of the analog meter MT shown in FIG. 7, the number of significant digits after the decimal point is 2.

[0136] For example, in step S29, when "30.04567..." is calculated as the pointer value and the number of significant digits after the decimal point is 2, the pointer value is determined as "30.0" up to the first decimal place. The value of the first decimal place is determined by rounding the value of the second decimal place.

[0137] In step S32, based on the image data obtained in step S2, the analog meter needle detection unit 116c generates data of a trimmed image (hereinafter referred to as trimmed image data) that trims the range including the pointer, the notation numbers in its vicinity, and the intersection points of the pointer and the circumference in the image of the photographed analog meter MT, and proceeds to step S33.

[0138] In step S33, the correction processing unit 116d displays the trimming image on the display unit 103 based on the trimming image data generated in step S32, instead of the image being displayed, and also displays the pointer value determined in step S27 or the pointer value determined in step S31 on the display unit 103. Then, the processing procedure proceeds to step S24.

[0139] FIG. 8 is a diagram showing an example of an interface for correcting the detected pointer value. At this time, an image (for example, a trimming image) including at least the range including the intersection of the pointer and the circumference in the image data of the analog meter MT is displayed, and a slide bar SL is displayed on the display unit 103 as a GUI functioning as a soft key. At this time, on the display unit 103, the pointer value may be displayed in a drum roll manner.

[0140] The drum roll display is an input interface for correcting the pointer value, and virtually displays, by graphics, drum rolls (number wheels) corresponding to the values of each digit of the pointer value. The drum roll can be operated through a touch panel (input unit 102) placed on the display unit 103.

[0141] For example, when the input unit 102 detects a flick input for rotating each drum roll up and down, the video of the drum roll is rotated and displayed in response, and the pointer value can be changed to an arbitrary value displayed accordingly.

[0142] The slide bar display is an input interface for setting a decimal point between the drum rolls, corresponds vertically to the positions of the drum rolls, and a decimal point can be set between the digits (drum rolls) by sliding the pointer of the slide bar SL left and right.

[0143] In the example shown in FIG. 8, for example, in response to sliding the position of the pointer of the slide bar SL left and right (in the horizontal direction of the screen), the position of the decimal point in the pointer value "52" can be changed. In FIG. 8, the position of the decimal point of the pointer value and the position of the pointer of the slide bar SL are set to coincide in the vertical direction (the vertical direction of the screen).

[0144] Note that the position of the pointer of the slide bar SL and the position of the decimal point in the numerical sequence represented by the drum roll do not necessarily have to coincide in the vertical direction. For example, for the display range of the numerical sequence of the pointer value, the slide bar SL may be displayed in a wider range than that, and the displayable range on the display unit 103 may be used to the maximum extent for decimal point input. In that case, the position of the decimal point is variable according to the relative position of the pointer within the movable range of the pointer of the slide bar SL.

[0145] In step S34, the correction processing unit 116d requests the inspector to input confirmation for the pointer value displayed on the display unit 103. Here, when an OK input operation is performed on the input unit 102, the displayed pointer value is determined as the measured value, and the processing procedure proceeds to step S36. On the other hand, when an NG input operation is performed on the input unit 102, the processing procedure proceeds to step S35.

[0146] In step S35, the correction processing unit 116d detects operations on the drum roll display and the slide bar display through the input unit 102. The correction processing unit 116d receives a correction instruction for the pointer value from the inspector, determines the corrected (displayed) pointer value as the measured value, and the processing procedure proceeds to step S36.

[0147] In step S36, the form creation unit 116a stores, as form data 106a in the storage unit 106, the image data obtained in step S2, the trimming image data generated in step S32, and the measured value determined in step S34 or step S35, in association with the identification information of the analog meter to be measured. After this, the processing procedure proceeds to step S37.

[0148] Subsequently, the analog meter needle inspection unit 116b instructs the display control unit 113 to display a prompt asking the inspector whether to also read the values indicated by other needles on the same analog meter MT (the analog meter MT in the image captured in step S2), and waits for the content input by the inspector to be transmitted from the input control unit 112 through the input unit 102 (step S37).

[0149] When the inspector inputs that the values indicated by other needles of the same analog meter are also to be read (step S37, Yes), the analog meter needle inspection unit 116b performs the needle value reading process again from step S3.

[0150] When the inspector inputs that the values indicated by other needles of the same analog meter are not to be read (step S37, No), the analog meter needle inspection unit 116b instructs the display control unit 113 to display a prompt asking the inspector whether all the needle values of the meters have been read (whether the work has been completed) on the display unit 103, and waits for the content input by the inspector to be transmitted from the input control unit 112 through the input unit 102 (step S38).

[0151] When the inspector inputs that the reading of the needle values of all the meters has not been completed (the needle value reading process for other meters is to be performed) (step S38, No), the analog meter needle inspection unit 116b returns to step S2, captures images of other meters, and performs the needle value reading process. When the inspector inputs that the reading of the needle values of all the meters has been completed (step S38, Yes), the analog meter needle inspection unit 116b ends the needle value reading process.

[0152] (Summary) As described above, in the information processing system of this embodiment, when the meter reader uses the portable terminal device 100 to meter an analog meter, the portable terminal device 100 performs pointer recognition in the mode selected by the meter reader based on the image data obtained by photographing the display panel of the analog meter, detects two numbers located on both sides so as to sandwich the pointer, and detects the metering value DT from the position of the intersection of the line connecting the two numbers and the pointer and the values m and n of the two numbers.

[0153] For example, since the above-described manual mode enables recognition of the pointer to be read, it is possible to avoid cases where it is difficult to recognize the pointer from the photographed image in the auto mode due to the installation environment of the meter or the like and thus metering cannot be performed, and cases where it is difficult to recognize the pointer from the photographed image in the auto mode when the shape, color, etc. of the pointer are different from the learned pointer, and additional learning has to be performed each time for corresponding to new pointer shapes and colors. Further, in the information processing system of this embodiment, when the analog meter to be read has a plurality of pointers, it is configured to be able to read the pointer values by all the pointers.

[0154] Therefore, according to the information processing system and the computer program having the above configuration, when the meter reader meters an analog meter having a plurality of pointers, it is possible to sequentially detect the pointer value of each of the plurality of pointers from the photographed image of the analog meter, and perform metering smoothly, so that the work load can be reduced.

[0155] Also, in the form data 106a, the metering value of the analog meter and the image data that is the source for detecting the metering value are stored in association with each other. Therefore, it can be confirmed by a person who later views the form data 106a. That is, according to this embodiment, it is possible to provide an information processing device, an information processing method, and a computer program that reduce the load of the metering work for reading an analog meter.

[0156] The computer program according to this embodiment may be transferred in a state stored in an electronic device, or may be transferred in a state not stored in the electronic device. In the latter case, the program may be transferred via a network, or may be transferred in a state stored in a storage medium. The storage medium is a non-transitory tangible medium. The storage medium is a computer-readable medium. The storage medium may be any medium that can store a program such as a CD-ROM or a memory card and is readable by a computer, regardless of its form.

[0157] Note that the present invention is not limited to the above-described embodiment as it is, and at the implementation stage, the components can be modified and embodied without departing from the gist thereof. Also, various inventions can be formed by appropriately combining a plurality of components disclosed in the above-described embodiment. Further, for example, a configuration in which some components are deleted from all the components shown in the embodiment is also conceivable. Furthermore, components described in different embodiments may be appropriately combined.

[0158] For example, in the above-described embodiment, an arc is drawn for the line connecting the two numbers M and N sandwiching the pointer, but the present invention is not limited thereto, and the line segments may be linearly connected to obtain the meter reading value.

[0159] Also, as an example of the analog meter, an analog meter having a circular outer shape as shown in FIG. 9 is exemplified, but the present invention is not limited thereto. FIGS. 9 and 10 are diagrams showing examples of analog meters that are the objects of the operation examples of the information processing apparatus according to the embodiment shown in FIGS. 3 to 5.

[0160] For example, an ammeter having a rectangular outer shape as shown in FIG. 9 may be used. FIG. 9(a) shows a rectangular direct current ammeter, FIG. 9(b) shows a rectangular ammeter capable of displaying negative values, FIG. 9(c) shows an alternating current ammeter with different scale intervals, and FIG. 9(d) shows a rectangular ammeter capable of displaying an extended region in addition to the normal region.

[0161] Moreover, it is not limited to an ammeter, and it may be an analog meter that measures other physical quantities or the like. For example, it may be a pressure gauge that measures positive pressure as shown in Fig. 10(a), a compound gauge that can measure and display both positive and negative pressure ranges as shown in Fig. 10(b), a vacuum gauge that measures negative pressure as shown in Fig. 10(c), and the like.

[0162] In addition, although Figs. 9 and 10 illustrate an analog meter with one pointer, an analog meter with multiple pointers can also be read in the same way as the circular-shaped one shown in the above-described embodiment.

[0163] In the above embodiment, the case where the portable terminal device 100 performs photographing of the analog meter, detection of the measured value of the analog meter, and creation of the form data 106a in which the detected value is recorded has been described as an example, but it is not limited thereto.

[0164] That is, the portable terminal device 100 may perform only the photographing of the analog meter, and the detection of the measured value of the analog meter and the creation of the form data 106a in which the detected value is recorded may be realized by information processing in the client terminal 300 or the server 400. In this case, the hardware and software described in the above embodiment are provided in the client terminal 300 or the server 400. Needless to say, various modifications can be made without departing from the gist of the present invention, and the present invention can be similarly implemented.

Description of Reference Numerals

[0165] 100…Portable terminal device, 101…Communication unit, 102…Input unit, 103…Display unit, 104…Camera, 106…Memory unit, 106a…Form data, 106b…Setting information data, 106c…Meter image data, 110…Control unit, 111…Communication control unit, 112…Input control unit, 113…Display control unit, 114…Image processing unit, 115…Position information processing unit, 116…Needle inspection data aggregation processing unit, 116a…Numerical data needle inspection unit, 116b…Analog meter needle inspection unit, 116c…Form creation unit, 116d…Correction processing unit, 200…Wireless base station, 300…Client terminal, 400…Server, 500…Network, Fq…Shooting guide frame, MT1~MTn…Meters, P…Intersection point, A1 - A4…Software keys.

Claims

1. A computer, based on image data obtained by photographing an analog meter that shows a measured value based on the positional relationship between a number and a pointer shown on a display panel, a first mode for detecting the pointer and recognizing the pointer to be read, and a second mode for recognizing the pointer to be read based on information input by a user in an image based on the image data, and a mode selection unit for selecting either one; recognizing the pointer to be read according to the mode selected by the mode selection unit, and based on the image data, a number detecting unit for detecting a first number located on one side surface of the pointer to be read and a second number located on the other side surface of the pointer to be read, respectively; a measured value detecting unit for detecting a measured value indicated by the pointer to be read based on the position of the first number, the position of the second number, the position of the pointer to be read, and the respective values of the first number and the second number A computer program for causing it to function.

2. The analog meter shows a measured value based on the positional relationship between a number shown on a display panel and each of a plurality of pointers, The mode selection unit, based on the image data obtained by photographing the analog meter, a first mode for detecting a plurality of the pointers and recognizing the pointer to be read among the plurality of the pointers, and a second mode for recognizing the pointer to be read based on information input by a user in an image obtained by photographing the analog meter, and selecting either one. The computer program according to claim 1.

3. The computer includes a storage unit that stores setting information in which either the first mode or the second mode is set as a standard mode, The mode selection unit determines whether to switch the mode from the standard mode, and selects the mode different from the standard mode in response to an input for switching the mode. The computer program according to claim 1.

4. The measured value detecting unit detects a measured value indicated by the pointer based on a ratio between a distance between the position of the first number and the position of the pointer to be read and a distance between the position of the second number and the position of the pointer, the value of the first number, and the value of the second number. The computer program according to claim 1.

5. A mode selection unit that selects either a first mode of detecting the pointer based on image data obtained by photographing an analog meter that indicates a measurement value based on the positional relationship between the numbers and the pointer written on the display panel and recognizing the pointer to be read, or a second mode of recognizing the pointer to be read based on information input by the user in the image based on the image data; A number detection unit that recognizes the pointer to be read according to the mode selected by the mode selection unit, and based on the image data, detects a first number located on one side surface of the pointer to be read and a second number located on the other side surface of the pointer to be read respectively; A measurement value detection unit that detects the measurement value indicated by the pointer to be read based on the positions of the first number, the position of the second number, the position of the pointer, and the respective values of the first number and the second number; An information processing apparatus comprising the above.

6. Based on image data obtained by photographing an analog meter that indicates a measurement value based on the positional relationship between the numbers and the pointer written on the display panel, select either a first mode of detecting the pointer and recognizing the pointer to be read, or a second mode of recognizing the pointer to be read based on information input by the user in the image based on the image data; Recognize the pointer to be read according to the selected mode, and based on the image data, detect a first number located on one side surface of the pointer to be read and a second number located on the other side surface of the pointer to be read respectively; Detect the measurement value indicated by the pointer to be read based on the positions of the first number, the position of the second number, the position of the pointer, and the respective values of the first number and the second number; An information processing method.

Images