Portable information terminal and image processing result registration method

Abstract

To provide a portable information terminal and an image processing result registration method that enable a user to easily confirm the necessity of registering results of image processing using a camera of the portable information terminal to a user application.SOLUTION: On the portable information terminal, a user application 87 determines the necessity of registering read data Dc in the user application 87 on the basis of the read data Dc as results of data processing received from a service part 88, and sends the determination result of the necessity of registration to the service part 88. Then, the service part 88 makes a displays 20 display an image view window showing a mark indicating the determination result received from the user application 87 superimposed on a code image.SELECTED DRAWING: Figure 2C

Description

[Technical Field]

[0001] The present invention relates to a technique for reading a code attached to a workpiece. [Background technology]

[0002] Patent Document 1 discloses an information reading device that reads codes such as barcodes and QR codes (registered trademarks). This information reading device acquires an image using a camera unit provided on the top end of a housing, and reads the code by decoding the code contained in the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-144178 Summary of the Invention [Problem to be solved by the invention]

[0004] There is also a need for a portable information terminal, such as a smartphone, to easily read the above codes and perform image processing such as OCR (Optical Character Recognition / Reader), which can be met by using the camera built into the portable information terminal to capture the image of the object to be processed.

[0005] In this type of image processing, the results of the image processing are sequentially registered in a predetermined user application for use. Conventionally, the user has to select the image processing results to be registered in the user application, which requires a lot of time and effort. In particular, the user cannot determine whether or not to register the results in the user application, which can make such a selection difficult for the user.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that allows a user to easily check whether or not the results of image processing using a camera on a mobile information terminal need to be registered in a user application. [Means for solving the problem]

[0007] The mobile information terminal of the present invention comprises a camera that captures an image of a workpiece and generates the captured image; an image processing unit that performs image processing based on the captured image; a service unit that has at least one input item and sends the results of the image processing to a user application that reflects the results of the image processing in the input item and receives a response from the user application; and a display unit that displays an image view window showing the captured image.The user application determines whether or not the results of the image processing need to be registered in the user application based on the results of the image processing received from the service unit and sends the determination result of whether or not registration is needed to the service unit.The service unit displays an image view window on the display unit that shows information indicating the determination result received from the user application superimposed on the captured image.

[0008] The image processing result registration method of the present invention comprises the steps of: a step of using a camera to capture an image of a workpiece and generate an captured image; a step of performing image processing based on the captured image; a step of a service unit sending the results of the image processing to a user application having at least one input item and reflecting the results of the image processing in the input item, and the service unit receiving a response from the user application; and a step of displaying an image view window showing the captured image on a display unit, wherein the user application determines whether or not the results of the image processing need to be registered in the user application based on the results of the image processing received from the service unit and sends the determination result of whether or not registration is needed to the service unit, and the service unit displays an image view window on the display unit showing information indicating the determination result received from the user application superimposed on the captured image.

[0009] In the present invention (portable information terminal and image processing result registration method) configured as described above, the user application determines whether or not the image processing results need to be registered in the user application based on the image processing results received from the service unit, and transmits the determination result on whether or not registration is needed to the service unit. The service unit then displays an image view window on the display unit, which shows information indicating the determination result received from the user application superimposed on the captured image. Therefore, the user can easily check whether or not the image processing results need to be registered based on the information indicating the determination result displayed in the image view window. In this way, the user can easily check whether or not the results of image processing using the camera of the portable information terminal need to be registered in the user application. [Effects of the Invention]

[0010] As described above, according to the present invention, the user can easily check whether or not the results of image processing using the camera of the portable information terminal need to be registered in a user application. [Brief explanation of the drawings]

[0011] [Figure 1A] 1 is a front view showing the external configuration of a portable information terminal according to the present invention; [Figure 1B] FIG. 1B is a rear view showing the external configuration of the mobile information terminal of FIG. 1A. [Figure 1C] FIG. 1B is a side view showing the external configuration of the mobile information terminal of FIG. 1A. [Figure 1D] FIG. 1B is a partial perspective view showing the external configuration of the mobile information terminal of FIG. 1A. [Figure 1E] FIG. 2 is a rear view showing the layout of the rear camera unit of the mobile information terminal. [Figure 1F] 1B is a side view schematically showing the operation of the rear camera unit of the mobile information terminal of FIG. 1A. [Figure 2A] FIG. 1B is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A. [Figure 2B] FIG. 1B is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A. [Figure 2C]FIG. 1B is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A. [Figure 3A] FIG. 10 is a diagram showing an example of a user application screen. [Figure 3B] FIG. 10 is a diagram showing an example of an image view window. [Figure 3C] FIG. 10 is a diagram showing an example of an image view window. [Figure 3D] 10A and 10B are diagrams showing an example of a display mode of a user application screen and an image view window. [Figure 4A] FIG. 10 is a diagram illustrating a state of alignment using a virtual aimer. [Figure 4B] FIG. 10 is a diagram schematically illustrating alignment using an aimer beam. [Figure 5A] 10 is a flowchart showing an example of a multiple code reading mode executed by the service unit. [Figure 5B] 5B is a flowchart showing an example of a registration necessity determination executed by a user application in accordance with the multiple code reading mode of FIG. 5A; [Figure 5C] 5B is a diagram illustrating the operations performed in the multiple code reading mode of FIG. 5A. FIG. [Figure 6A] 10 is a flowchart showing an example of code reading executed by a portable information terminal. [Figure 6B] 6B is a flowchart showing an example of a single code reading mode executed in the code reading of FIG. 6A. [Figure 6C] 6B is a flowchart showing an example of a multiple code reading mode executed in the operation of FIG. 6A. [Figure 6D] 6B is a flowchart showing an example of a data saving process executed in the code reading of FIG. 6A. [Figure 6E] 10 is a flowchart showing an example of screenshot control. [Figure 7A] 10 is a flowchart showing an example of code reading condition control. [Figure 7B] FIG. 7B is a diagram schematically showing the operation executed in the code reading condition control of FIG. 7A. [Figure 7C] FIG. 7B is a diagram schematically showing the operation executed in the code reading condition control of FIG. 7A. [Figure 7D] FIG. 7B is a diagram schematically showing the operation executed in the code reading condition control of FIG. 7A. [Figure 8A] FIG. 10 is a rear view showing an example of an attachment. [Figure 8B] FIG. 10 is a rear view showing an example of an attachment. [Figure 8C] FIG. 10 is a rear view showing an example of an attachment. [Figure 8D] FIG. 10 is a rear view showing an example of an attachment. [Figure 8E] FIG. 10 is a rear view showing an example of an attachment. [Figure 9] FIG. 2 is a block diagram showing the electrical configuration of a portable information terminal that determines the type of attachment. [Figure 10] 10 is a flowchart showing a modified example of the security mode setting method. [Figure 11A] FIG. 10 is a rear view showing a modified example of the layout of the rear camera unit. [Figure 11B] FIG. 10 is a rear view showing a modified example of the layout of the rear camera unit. [Figure 11C] FIG. 10 is a rear view showing a modified example of the layout of the rear camera unit. [Figure 11D] FIG. 10 is a rear view showing a modified example of the layout of the rear camera unit. [Figure 12A] FIG. 10 is a diagram showing a first modified example of the display mode of a user application screen and an image view window. [Figure 12B] FIG. 10 is a diagram showing a first modified example of the display mode of a user application screen and an image view window. [Figure 13A] FIG. 10 is a diagram showing a second modified example of the display mode of a user application screen and an image view window. [Figure 13B] FIG. 10 is a diagram showing a second modified example of the display mode of a user application screen and an image view window. [Figure 14A] 10 is a flowchart showing a first modified example of display control of an image view window. [Figure 14B] 10 is a flowchart showing a second modified example of display control of the image view window. [Figure 15] 10 is a flowchart showing an example of registration management of read data by a user application. DETAILED DESCRIPTION OF THE INVENTION

[0012] Fig. 1A is a front view showing the external configuration of a mobile information terminal according to the present invention, Fig. 1B is a rear view showing the external configuration of the mobile information terminal of Fig. 1A, Fig. 1C is a side view showing the external configuration of the mobile information terminal of Fig. 1A, Fig. 1D is a partial perspective view showing the external configuration of the mobile information terminal of Fig. 1A, Fig. 1E is a rear view showing the layout of the rear camera unit of the mobile information terminal, and Fig. 1F is a side view schematically showing the operation of the rear camera unit of the mobile information terminal of Fig. 1A. In this embodiment, the transverse direction Xs, longitudinal direction Xl, and thickness direction Xt of the mobile information terminal 1, which are orthogonal to each other, are appropriately shown.

[0013] The mobile information terminal 1 has a rectangular housing 11 with rounded corners when viewed from the front or rear in the thickness direction Xt. The housing 11 is short in the short-side direction Xs and long in the long-side direction Xl; in other words, the size of the housing 11 in the long-side direction Xl is longer than the size of the housing 11 in the short-side direction Xs. The housing 11 has a front surface 12 located in front of the thickness direction Xt, a back surface 13 located behind the thickness direction Xt, a right side surface 14 located on the right side when viewed from the front, a left side surface 15 located on the left side when viewed from the front, a top surface 16 located on the upper side when viewed from the front, and a bottom surface 17 located on the lower side when viewed from the front. The front surface 12 and the back surface 13 are parallel to each other and face each other with a gap in the thickness direction Xt. The right end of the front surface 12 and the right end of the back surface 13 are connected by a right side surface 14 provided in the thickness direction Xt, the left end of the front surface 12 and the left end of the back surface 13 are connected by a left side surface 15 provided in the thickness direction Xt, the upper end of the front surface 12 and the upper end of the back surface 13 are connected by an upper surface 16 provided in the thickness direction Xt, and the lower end of the front surface 12 and the lower end of the back surface 13 are connected by a lower surface 17 provided in the thickness direction Xt.

[0014] The mobile information terminal 1 includes a display 20 attached to a housing 11. The display 20 is a touch panel display provided on a front surface 12 of the housing 11, and displays a screen to a user who uses the mobile information terminal 1 and receives input in response to touch operations on the display 20.

[0015] The mobile information terminal 1 also has volume buttons 21, 22 and a trigger button 23 arranged on the right side surface 14 of the housing 11. The right side surface 14 of the housing 11 has a recess 141 recessed toward the inside of the housing 11. This recess 141 extends in the longitudinal direction Xl. The volume buttons 21, 22 and the trigger button 23 are arranged in the recess 141 and aligned in this order from top to bottom in the longitudinal direction Xl. The volume buttons 21, 22 and the trigger button 23 are arranged above the center M11 of the housing 11 in the longitudinal direction Xl. When the volume button 21 is operated by a user, the volume of the speaker of the mobile information terminal 1 is increased, and when the volume button 22 is operated by a user, the volume of the speaker of the mobile information terminal 1 is decreased. When the trigger button 23 is operated by a user, it triggers the mobile information terminal 1 to start reading a code. The trigger button 23 has a base portion 231 disposed within the recess 141 and extending in the longitudinal direction Xl, and a protrusion portion 232 protruding from the base portion 231 to the outside of the recess 141. In the longitudinal direction Xl, the protrusion portion 232 is shorter than the base portion 231 and is provided in the center of the base portion 231. The base portion 231 protrudes outward (to the right) from the right side surface 14 (FIGS. 1A and 1B).

[0016] Furthermore, the mobile information terminal 1 includes a camera module 3 attached to the housing 11. The camera module 3 has a front camera 31 provided on the front surface 12 of the housing 11 and a rear camera unit 32 provided on the back surface 13 of the housing 11.

[0017] The front camera 31 has a camera field of view (forward field of view) that extends forward from the front surface 12 of the housing 11, and captures an image of this field of view using a rolling shutter method. Note that FIG. 1A shows the objective lens of the front camera 31 that appears on the front surface 12 of the housing 11. The front camera 31 is an autofocus camera that automatically adjusts the focus position. This autofocus is performed by adjusting the focus position based on the brightness histogram of the image captured by the front camera 31. The front camera 31 may also be a fixed-focus camera.

[0018] The rear camera unit 32 is provided to protrude rearward from the rear surface 13 of the housing 11 in the thickness direction Xt and includes a rear camera 33, a wide-angle illumination unit 34, a narrow-angle illumination unit 35, an aimer illumination unit 36, and a distance measurement sensor 37. The rear camera 33 has a camera field of view V33 (rear field of view) extending rearward from the rear surface 13 of the housing 11 and captures the field of view using a rolling shutter system. The rear camera 33 is, for example, a color camera capable of generating color images. Note that FIG. 1B shows the objective lens of the rear camera unit 32 appearing on the rear surface 13 of the housing 11. The rear camera unit 32 is an autofocus camera that automatically adjusts the focus position. When the rear camera unit 32 is used for photography, autofocus is performed based on a brightness histogram, as with the front camera 31. When the rear camera unit 32 is used for code reading, autofocus is performed based on the detection results of the distance measurement sensor 37, as described below. The wide-angle illumination unit 34 irradiates wide-angle illumination light L34, which is illumination light with a relatively wide light distribution angle, from the back surface 13 of the housing 11 toward the rear. The narrow-angle illumination unit 35 irradiates narrow-angle illumination light L35, which is illumination light with a narrower light distribution angle than the wide-angle illumination light L34, from the back surface 13 of the housing 11 toward the rear. The aimer light illumination unit 36 ​​irradiates aimer light L36, which is visible light, from the back surface 13 of the housing 11 toward the rear. The distance measurement sensor 37 is, for example, a TOF (Time of Flight) sensor, and measures the distance to the measurement object based on the time from when distance measurement light L37 is irradiated from the back surface 13 of the housing 11 toward the rear until distance measurement light L37 reflected by the measurement object is detected.

[0019] 1E, the wide-angle illuminator 34, the distance measurement sensor 37, the rear camera 33, and the aimer light illuminator 36 are arranged in this order in the short-side direction Xs. That is, in the short-side direction Xs, the aimer light illuminator 36 is located on one side of the rear camera 33, the distance measurement sensor 37 and the wide-angle illuminator 34 are located on the other side of the rear camera 33, the rear camera 33 is located between the distance measurement sensor 37 and the aimer light illuminator 36, and the distance measurement sensor 37 is located between the rear camera 33 and the wide-angle illuminator 34. The narrow-angle illuminator 35 is located below the rear camera 33 in the long-side direction Xl. The optical axes of the rear camera 33, the wide-angle illuminator 34, the narrow-angle illuminator 35, the aimer light illuminator 36, and the distance measurement sensor 37 extend in the thickness direction Xt and are parallel to each other. The optical axis of the rear camera 33 is separated by a distance K34 from the optical axis of the wide-angle illumination unit 34, the optical axis of the rear camera 33 is separated by a distance K35 from the optical axis of the narrow-angle illumination unit 35, the optical axis of the rear camera 33 is separated by a distance K36 from the optical axis of the aimer light illumination unit 36, and the optical axis of the rear camera 33 is separated by a distance K37 from the optical axis of the distance measurement sensor 37. Here, the optical axis of the distance measurement sensor 37 is a virtual axis extending parallel to the optical axis of the optical system that emits distance measurement light L37 and the optical axis of the optical system that collects the distance measurement light L37 on the optical sensor, at a position midway between these optical axes. The distance K34 between the rear camera 33 and the wide-angle illumination unit 34 is longer than the distance K35 between the rear camera 33 and the narrow-angle illumination unit 35, the distance K36 between the rear camera 33 and the aimer light illumination unit 36, and the distance K37 between the rear camera 33 and the distance measurement sensor 37.

[0020] As shown in FIG. 1F , the rear camera 33 has a camera field of view V33 that extends rearward in the short-side direction Xs. The wide-angle illuminator 34 irradiates the camera field of view V33 with wide-angle illumination light L34, and the narrow-angle illuminator 35 irradiates the camera field of view V33 with narrow-angle illumination light L35. The light distribution angle of the narrow-angle illumination light L35 is smaller than that of the wide-angle illumination light L34, and the illumination range of the narrow-angle illumination light L35 is included in the illumination range of the wide-angle illumination light L34. The aimer light illuminator 36 irradiates the camera field of view V33 with aimer light L36, and the distance measurement sensor 37 irradiates the camera field of view V33 with distance measurement light L37. The aimer light L36 and the distance measurement light L37 are irradiated approximately at the center of the camera field of view V33, and the illumination ranges of the aimer light L36 and the distance measurement light L37 are included in the illumination ranges of the wide-angle illumination light L34 and the narrow-angle illumination light L35, respectively.

[0021] FIG. 2A is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A, mainly showing the configuration for controlling the camera module. Mobile information terminal 1 includes a control unit 5 that controls camera module 3, a volatile storage unit 61, and a nonvolatile storage unit 62. Control unit 5 is, for example, a processor, and controls camera module 3 and data storage in volatile storage unit 61 and nonvolatile storage unit 62. Volatile storage unit 61 is, for example, a random access memory (RAM) that stores data in a volatile manner. Nonvolatile storage unit 62 is, for example, a solid state drive (SSD) or hard disk drive (HDD) that stores data in a nonvolatile manner. Mobile information terminal 1 also includes a photography processing unit 71 that causes control unit 5 to take a photo using camera module 3, and a code reading processing unit 72 that causes control unit 5 to read a code using camera module 3. The photography processing unit 71 and the code reading processing unit 72 are processes generated by a processor installed in mobile information terminal 1 executing a predetermined program.

[0022] The photography processing unit 71 stores in the nonvolatile storage unit 62 setting values ​​(photography setting values) that are set in the camera module 3 to take a photo using the front camera 31 or the rear camera 33. Examples of the photography setting values ​​include the exposure time of the front camera 31 or the rear camera 33, the brightness of the wide-angle illumination light L34 that is irradiated from the wide-angle illumination unit 34 into the camera field of view V33, and the focus position of the front camera 31 or the rear camera 33. The code reading processing unit 72 stores in the nonvolatile storage unit 62 setting values ​​(code reading setting values) that are set in the camera module 3 to read a code using the rear camera 33. Examples of the code reading setting values ​​include the exposure time of the rear camera 33, the brightness of the narrow-angle illumination light L35 that is irradiated from the narrow-angle illumination unit 35 into the camera field of view V33, the focus position of the rear camera 33, and the size of the code to be read.

[0023] The control unit 5 has a camera control public API 51, a camera control private API 52, an API execution switching control unit 53, and a camera control unit 54, and the camera control unit 54 has a public API control unit 55 and a private API control unit 56. These components of the control unit 5 are processes generated when a processor mounted on the mobile information terminal 1 executes a predetermined program.

[0024] When the camera control public API 51 receives a photography instruction from the photography processing unit 71 to instruct the execution of photography, the API execution switching control unit 53 manages the execution of photography based on the determination result of whether security mode is set. The security mode is set by an administrator's operation on the security setting unit 74, which will be described later. If security mode is not set (in other words, if normal mode is set), the API execution switching control unit 53 permits photography and sends a photography instruction to the public API control unit 55. Therefore, the public API control unit 55 reads photography setting values ​​from the nonvolatile storage unit 62 and operates the camera module 3 based on these photography setting values. That is, the public API control unit 55 acquires a photographic image Ip (captured image) captured by the front camera 31 or the rear camera 33 within the respective camera fields of view and stores it in the nonvolatile storage unit 62. On the other hand, if security mode is set, the API execution switching control unit 53 prohibits photography and does not send a photography instruction to the public API control unit 55. Therefore, the public API control unit 55 does not perform photography using the camera module 3.

[0025] When the camera control private API 52 receives a code reading instruction from the code reading processing unit 72 to execute code reading, the API execution switching control unit 53 transmits the code reading instruction to the private API control unit 56. Therefore, the private API control unit 56 reads out the code reading setting value from the nonvolatile storage unit 62 and operates the camera module 3 based on the code reading setting value. That is, the private API control unit 56 acquires a code image Ic (captured image) obtained by capturing the camera field of view V33 with the rear camera 33 and stores the code image Ic in the volatile storage unit 61. As will be described later, a decoding process is performed on the code image Ic stored in the volatile storage unit 61. Furthermore, if the administrator has set nonvolatile storage of the code image Ic, the private API control unit 56 stores the code image Ic in the nonvolatile storage unit 62. However, if the administrator has not set nonvolatile storage of the code image Ic, the private API control unit 56 does not store the code image Ic in the nonvolatile storage unit 62.

[0026] The mobile information terminal 1 also includes an SoC (System on Chip) 73. The SoC 73 has an ISP (Image Signal Processor) 731, a DSP (Digital Signal Processor) 732, a GPU (Graphics Processing Unit) 733, and a CPU (Central Processing Unit) 734. The ISP 731 receives the photographic image Ip or the code image Ic from the front camera 31 or the rear camera 33 of the camera module 3 and performs corrections specific to the front camera 31 or the rear camera 33 on the photographic image Ip or the code image Ic. The camera-specific corrections correspond to corrections of errors caused by the characteristics of the camera's optical system or image sensor. The SoC 73 then performs appropriate signal processing on the photographic image Ip or the code image Ic corrected by the ISP 731 using the DSP 732, the GPU 733, or the CPU 734. The public API control unit 55 and the private API control unit 56 store the photographic image Ip or the code image Ic that has undergone signal processing by the SoC 73 in the non-volatile storage unit 62 or the volatile storage unit 61 .

[0027] The mobile information terminal 1 also includes a security setting unit 74, a disclosure prohibition unit 75, a screenshot prohibition unit 76, and a screenshot control unit 77. The security setting unit 74 displays an administrator setting screen on the display 20 and accepts setting operations on the display 20 by the administrator. Specifically, the security setting unit 74 sets the security mode when the administrator performs an operation on the administrator setting screen to instruct setting the security mode, and cancels the security mode when the administrator performs an operation on the administrator setting screen to instruct canceling the security mode. Note that the security setting unit 74 displays the administrator setting screen on the display 20 when it confirms that a predetermined password has been entered on the display 20. When the security mode is set, the disclosure prohibition unit 75 prohibits the disclosure of the photo image Ip or the code image Ic captured by the front camera 31 or the rear camera 33.

[0028] When security mode is set, the screenshot prohibition unit 76 prohibits taking a screenshot of the display 20. When taking a screenshot is prohibited by the screenshot prohibition unit 76, the screenshot control unit 77 ignores the instruction to take a screenshot and does not take a screenshot. Instead of prohibiting taking a screenshot, the screenshot control unit 77 may invalidate (replace all pixel values ​​with a fixed pixel value) or mask (replace with a mask consisting of fixed pixel values) the screenshot image of the display 20 and store the masked image in the non-volatile storage unit 62.

[0029] Furthermore, the mobile information terminal 1 has a position information calculation unit 78. The position information calculation unit 78 uses a Global Navigation Satellite System (GNSS) to calculate the position information of the mobile information terminal 1. The position information of the mobile information terminal 1 calculated by the position information calculation unit 78 is stored in the volatile storage unit 61.

[0030] 2B is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A, and mainly shows a configuration for controlling communication between mobile information terminal 1 and the outside. Mobile information terminal 1 has a communication control unit 81, an Internet communication unit 82, a telephone communication unit 83, and a call recording unit 84. Internet communication unit 82 communicates with external servers and the like via the Internet, telephone communication unit 83 makes calls to external telephone terminals via communication lines (including telephone lines, LAN lines, and Internet lines), and call recording unit 84 stores audio data of calls made by telephone communication unit 83 in non-volatile storage unit 62. Furthermore, the operations of Internet communication unit 82, telephone communication unit 83, and call recording unit 84 are controlled based on the determination result by communication control unit 81 of whether security mode is set.

[0031] That is, when communication control unit 81 determines that security mode is not set, it permits Internet connection by Internet communication unit 82. When communication control unit 81 determines that security mode is set, it prohibits Internet connection by Internet communication unit 82.

[0032] The communication control unit 81 also has a registered telephone number list that indicates registered telephone numbers. If the communication control unit 81 determines that the security mode is not set, it allows the telephone communication unit 83 to make calls to both telephone terminals having telephone numbers included in the registered telephone number list and telephone terminals having telephone numbers not included in the registered telephone number list. If the communication control unit 81 determines that the security mode is set, it allows calls between the telephone communication unit 83 and telephone terminals having telephone numbers included in the registered telephone number list, but prohibits calls between the telephone communication unit 83 and telephone terminals having telephone numbers not included in the registered telephone number list.

[0033] Furthermore, when the communication control unit 81 determines that the security mode is not set, it allows the call recording unit 84 to save the voice data in the nonvolatile storage unit 62. When the communication control unit 81 determines that the security mode is set, it prohibits the call recording unit 84 from saving the voice data in the nonvolatile storage unit 62.

[0034] 2C is a block diagram showing the electrical configuration of the mobile information terminal of FIG. 1A, and mainly shows the configuration for controlling code reading by mobile information terminal 1. Mobile information terminal 1 has a decoder 86 (image processing unit), a user application 87, and a service unit 88. Each of the decoder 86, user application 87, and service unit 88 is a process generated by, for example, a processor mounted on mobile information terminal 1 executing a predetermined program. The service unit 88 functions as a backend that coordinates the operations of the decoder 86, user application 87, code reading processing unit 72, trigger button 23, and display 20.

[0035] When the service unit 88 confirms that the trigger button 23 has been pressed by the user, it transmits a code reading instruction to the code reading processing unit 72, and the code reading processing unit 72 outputs the code reading instruction received from the service unit 88 to the camera control private API 52. In response to receiving the code reading instruction from the camera control private API 52, the camera module 3 acquires the code image Ic using the procedure described above. That is, the private API control unit 56 sets the code reading setting values ​​read from the non-volatile storage unit 62 in the camera module 3. Then, the private API control unit 56 causes the rear camera 33 to capture an image of the camera field of view V33 while irradiating narrow-angle illumination light L35 from the narrow-angle illumination unit 35. At this time, the private API control unit 56 adjusts the focus position of the rear camera 33 to the position of the code on the workpiece based on the result of detecting the distance to the code (specifically, the workpiece to which the code is attached) measured by the distance measurement sensor 37. When the rear camera 33 is focused on the workpiece in this manner, the depth of field of the rear camera 33 includes the irradiation positions of the aimer light L36 and the distance measurement light L37 on the workpiece W. Then, the non-public API control unit 56 causes the rear camera 33 to capture an image. As a result, an image of the entire camera field of view V33 including the code is acquired as a code image Ic, and the code image Ic is stored in the volatile storage unit 61. At this time, the non-public API control unit 56 repeatedly acquires the code image Ic at a predetermined frame period and overwrites and stores it in the volatile storage unit 61. Therefore, only the latest code image Ic is stored in the volatile storage unit 61.

[0036] Then, the service unit 88 requests the decoder 86 to execute the decoding process when a predetermined waiting time (for example, one second) has elapsed since it confirmed that the trigger button 23 has been pressed by the user. As a modified example, the service unit 88 may request the decoder 86 to execute the decoding process when the trigger button 23 is pressed by the user (i.e., without any waiting time). Upon receiving the request from the service unit 88, the decoder 86 reads out the code image Ic stored in the volatile storage unit 61 and executes the decoding process on the code image Ic. As a result, the read data Dc indicated by the code image Ic is read by the decoder 86. This read data Dc is, for example, text data.

[0037] The decoder 86 transmits the read data Dc to the service unit 88, and the service unit 88 transmits the read data Dc received from the service unit 88 to the user application 87. The user application 87 inputs the read data Dc received from the decoder 86 into predetermined input fields. At this time, the user application 87 displays a user application screen 91 (FIG. 3A) showing input fields on the display 20 to assist the user in inputting the read data Dc.

[0038] FIG. 3A is a diagram showing an example of a user application screen. The user application screen 91 of FIG. 3A is displayed on the display 20 as the user application 87 is executed. The user application screen 91 has multiple input item fields 911, each showing an input item A1, A2, ..., and multiple data display fields 912, each showing the read data Dc entered in the multiple input items A1, A2, .... In the example of FIG. 3A, read data Dc1 and Dc2 have been entered in the input items A1 and A2, respectively, while read data Dc has not yet been entered in the input items A3, A4, .... The user application 87 focuses on the input item A3 selected by the user's operation on the display 20. When the user presses the trigger button 23 in this state, the read data Dc3 is acquired by the service unit 88 and transmitted to the user application 87 according to the procedure described above. The user application 87 then inputs the read data Dc3 received from the service unit 88 into the focused input item A3. As a result, on the user application screen 91, the read data Dc3 is displayed in the data display field 912 corresponding to the input item A3.

[0039] The service unit 88 also displays an image view window 93 (FIGS. 3B and 3C) on the display 20 to assist the user in reading the code. FIGS. 3B and 3C are diagrams showing examples of image view windows. The image view window 93 has a rectangular shape and displays the code image Ic stored in the volatile storage unit 61. The image view window 93 is an image view window (also called a live view window) that periodically updates images by capturing images. Alternatively, the image view window 93 may update the screen to display the read image when the decoder 86 successfully reads the code. The image view window 93 is displayed on the display 20 by the service unit 88 when the rear camera 33 of the camera module 3 is activated. Therefore, for example, the service unit 88 displays the image view window 93 on the display 20 in response to pressing of the trigger button 23. That is, upon confirming that the trigger button 23 has been pressed, the service unit 88 transmits a code reading instruction to the code reading processing unit 72. When this code reading instruction is received by the code reading processing unit 72, the rear camera 33 is activated for code reading in the above-described procedure, and the display of the image view window 93 on the display 20 begins.

[0040] As described above, when the trigger button 23 is pressed by the user, the code image Ic is acquired at a predetermined frame period and overwritten and saved in the volatile storage unit 61. In response to this, the service unit 88 reads out the code image Ic saved in the volatile storage unit 61 at the frame period, acquires an image of the central part of the code image Ic as an image view image Il, and displays it in the image view window 93 being displayed on the display 20. In the examples of FIGS. 3B and 3C, the code C included in the camera field of view V33 is displayed in the image view image Il.

[0041] Furthermore, the service unit 88 displays a virtual aimer 94 in the center of the image view window 93. Therefore, the user holding the mobile information terminal 1 can align the virtual aimer 94 displayed in the image view image Il with the code C. As a result, the virtual aimer 94, which is shifted from the code C as shown in FIG. 3B, can be aligned with the code C as shown in FIG. 3C. This allows the code C to be positioned in the center of the camera field of view V33. Then, when the decoder 86 completes the decoding process for the code image Ic, the service unit 88 ends the display of the image view image Il on the display 20. Note that the display of the image view image Il begins simultaneously with confirmation that the trigger button 23 has been pressed, while the decoding process by the decoder 86 begins after a waiting time has elapsed since confirmation that the trigger button 23 has been pressed. Therefore, the image view image Il is continuously displayed on the display 20 for at least the waiting time.

[0042] The service unit 88 also displays an image view window 93 on the display 20 together with the user application screen 91 displayed by the user application 87 ( FIG. 3D ). Here, FIG. 3D is a diagram showing an example of the display mode of the user application screen and the image view window. As shown in FIG. 3D , the service unit 88 displays the image view window 93 overlapping the user application screen 91. That is, the image view window 93 is displayed in front of the user application screen 91. At this time, the service unit 88 displays the image view window 93 semi-transparently. Therefore, the overlapping portion of the user application screen 91 with the image view window 93 is displayed semi-transparently. That is, the overlapping portion is displayed on the display 20 so that it can be seen by the user. In the example of FIG. 3D , input item A3 focused by the user is included in the overlapping portion, but is visible to the user because it is displayed semi-transparently through the image view window 93.

[0043] At this time, the service unit 88 adjusts the transparency, which is the degree to which the image view window 93 transmits the user application screen 91. Specifically, the service unit 88 calculates a representative value that represents the saturation of the image in the image view window 93. For example, the maximum value, average value, or median value of the saturation of the image in the image view window 93 is calculated as the representative value. The service unit 88 then sets a higher transparency as the saturation (representative value) of the image displayed in the image view window 93 increases.

[0044] In addition, together with the display of the virtual aimer 94 on the image view image Il, the aimer light L36 is also emitted by the aimer light illuminating unit 36. That is, when the private API control unit 56 receives a code reading instruction from the code reading processing unit 72, it causes the aimer light illuminating unit 36 ​​to emit the aimer light L36. Therefore, as described above, the user can align the camera field of view V33 with the code C using the virtual aimer 94, and can also align the camera field of view V33 with the code C using the aimer light L36 (in other words, real aim) irradiated onto the workpiece to which the code C is attached (FIGS. 4A and 4B).

[0045] FIG. 4A is a diagram schematically illustrating alignment using a virtual aimer, and FIG. 4B is a diagram schematically illustrating alignment using an aimer light. In both the examples of FIG. 4A and FIG. 4B, the aimer light L36 emitted from the aimer light illumination unit 36 ​​is irradiated onto the workpiece W to which the code C is attached. However, in the example of FIG. 4A, the user's line of sight E is hidden by the mobile information terminal 1 and does not reach the aimer light L36 on the workpiece W. Therefore, the user cannot see the aimer light L36 and performs alignment based on the virtual aimer 94 displayed on the display 20. On the other hand, in the example of FIG. 4B, the user's line of sight E is not hidden by the mobile information terminal 1 and reaches the aimer light L36 on the workpiece W. Therefore, even if the user cannot see the display 20, the user can see the aimer light L36 and perform alignment based on the aimer light L36 irradiated onto the workpiece W.

[0046] 3B and 3C, the mobile information terminal 1 can execute a single code reading mode in which a single code C is read, and a multiple code reading mode in which a plurality of codes C are simultaneously read. Next, the multiple code reading mode will be described.

[0047] Fig. 5A is a flowchart showing an example of a multiple code reading mode executed by the service unit, Fig. 5B is a flowchart showing an example of a registration necessity determination executed by a user application in accordance with the multiple code reading mode of Fig. 5A, and Fig. 5C is a diagram schematically showing the operation executed in the multiple code reading mode of Fig. 5A. Fig. 5C illustrates an example of an operation for reading six codes C1 to C6 attached to a work W, which is a document. In this example, read data Dc, which is the result of reading code C, is registered in a data management table 95 possessed by user application 87.

[0048] This data management table 95 indicates, for each of six read targets Tx1 to Tx6, data registration conditions 951 that the read data Dc must satisfy and read results 952 to which the read data Dc will be registered. The data registration conditions 951 are composed of a combination of a code type, a character string, and a position. The code type indicates a code type condition required for the type of code C corresponding to the read data Dc, the character string indicates a character string condition required for the character string included in the read data Dc, and the position indicates a position condition required for the position of the code C corresponding to the read data Dc on the workpiece W. In this example, the position condition is N / C (No Care) and is therefore invalid. Incidentally, the conditions included in the data registration conditions 951 are not limited to this example. The data registration conditions 951 may include, for example, a numerical condition required for a numerical value included in the read data Dc.

[0049] In step S101, the service unit 88 causes the decoder 86 to perform a decoding process on the code image Ic obtained by capturing an image of the code C included in the image view window 93, thereby acquiring read data Dc of the code C. In the example shown in the "S101 (first time)" column of FIG. 5C, four codes C1 to C4 out of six codes C1 to C6 are included in the image view window 93. Therefore, a code image Ic including the four codes C1 to C4 is acquired and stored in the volatile storage unit 61, and the decoder 86 performs a decoding process on each image of the codes C1 to C4 included in the code image Ic. As a result, the service unit 88 acquires four pieces of read data Dc1 to Dc4 corresponding to the four codes C1 to C4, respectively.

[0050] In step S102, the service unit 88 transmits the four read data Dc1 to Dc4 acquired in step S101 to the user application 87. In response to this, the user application 87 executes the registration necessity determination shown in Fig. 5B. Note that the user application 87 executes the registration necessity determination for each of the four read data Dc1 to Dc4, but here, the registration necessity determination for one read data Dc will be described.

[0051] When the user application 87 receives the read data Dc from the service unit 88 (step S201), it compares the read data Dc with the data management table 95 and determines whether or not the read data Dc needs to be registered in the data management table 95 (steps S202 to S205). In step S202, the user application 87 checks whether or not there is a data registration condition 951 that is satisfied by the read data Dc among the data registration conditions 951 for each of the read targets Tx1 to Tx6. If there is no corresponding data registration condition 951 ("NO" in step S202), the user application 87 determines that the read data Dc is not a read target and therefore does not need to be registered, and transmits the determination result to the service unit 88 (step S205).

[0052] If the corresponding data registration condition 951 exists (if "YES" in step S202), the user application 87 determines whether the read data Dc has already been registered in the read result 952 corresponding to the corresponding data registration condition 951 (step S203). If the read data Dc has already been registered (if "YES" in step S203), the user application 87 determines that the read data Dc has already been registered and therefore does not need to be registered, and transmits the determination result to the service unit 88 (step S205). If the read data Dc has not already been registered (if "NO" in step S203), the user application 87 determines that the read data Dc needs to be registered, and transmits the determination result to the service unit 88 (step S204).

[0053] 5C, it is determined that each of the read data Dc1 to Dc4 satisfies the data registration condition 951 for the read targets Tx1 to Tx4 (YES in step S202). In addition, since the read results 952 corresponding to each of the read targets Tx1 to Tx4 are empty and the read data Dc1 to Dc4 are not yet registered, it is determined that the read data Dc1 to Dc4 need to be registered (steps S203 and S204).

[0054] 5A, when the service unit 88 receives the necessity determination result from the user application 87 (step S103), it marks the codes C1 to C4 in accordance with the determination result in the image view window 93 (step S104). In this example, since it has been determined that the codes C1 to C4 (in other words, the corresponding read data Dc1 to Dc4) require registration, the service unit 88 displays a mark Ma (for example, a black check mark) indicating that registration is required for the codes C1 to C4 (the column "S104 (first time)" in FIG. 5C).

[0055] When the service unit 88 confirms a user's flick operation (registration instruction operation) on the display 20 displaying the image view window 93 (step S105), it transmits a registration instruction to the user application 87 to register the read data Dc1 to Dc4 determined to require registration in the read result 952 of the data management table 95 (step S106). Upon receiving the registration instruction from the service unit 88, the user application 87 registers the read data Dc1 to Dc4 indicated by the registration instruction in the read result 952 of the corresponding read targets Tx1 to Tx4 (the column "Registration (first time)" in FIG. 5C).

[0056] In the example shown in the "S101 (2nd time)" column of Figure 5C, of ​​the six codes C1 to C6, four codes C3 to C6 are included in the image view window 93, and the service unit 88 acquires four read data Dc3 to Dc6 corresponding to the four codes C3 to C6, respectively.

[0057] In step S102, the service unit 88 transmits the four read data Dc3 to Dc6 acquired in step S101 to the user application 87. In response to this, the user application 87 executes the registration necessity determination in Fig. 5B. In the example shown in the column "S102 (second time)" in Fig. 5C, it is determined that each of the read data Dc3 to Dc6 satisfies the data registration condition 951 for the read targets Tx3 to Tx6 ("YES" in step S202).

[0058] However, since the read data Dc3 and Dc4 have already been registered in the read result 952 corresponding to the read targets Tx3 and Tx4, respectively, it is determined that the read data Dc3 and Dc4 do not need to be registered (steps S203 and S205). Therefore, a determination result that the read data Dc3 and Dc4 have already been registered and therefore do not need to be registered is transmitted from the user application 87 to the service unit 88. On the other hand, since the read result 952 corresponding to the read targets Tx5 and Tx6, respectively, is empty and the read data Dc5 and Dc6 have not yet been registered, it is determined that the read data Dc5 and Dc6 need to be registered (steps S203 and S204). Therefore, a determination result that the read data Dc3 and Dc4 need to be registered is transmitted from the user application 87 to the service unit 88.

[0059] As shown in FIG. 5A, when the service unit 88 receives the necessity determination result from the user application 87 (step S103), it attaches marks corresponding to the determination result to the codes C3 to C6 in the image view window 93. In this example, it is determined that the codes C3 and C4 (in other words, the corresponding read data Dc3 and Dc4) have already been registered and therefore do not need to be registered. Therefore, the service unit 88 displays a mark Mb (for example, a white check mark) indicating that the codes C3 and C4 have been registered (the column "S104 (second time)" in FIG. 5C). Furthermore, it is determined that the codes C5 and C6 (in other words, the corresponding read data Dc5 and Dc6) need to be registered. Therefore, the service unit 88 displays a mark Ma indicating that the codes C5 and C6 need to be registered (the column "S104 (second time)" in FIG. 5C).

[0060] 5C shows a mark Ma indicating that registration is necessary and a mark Mb indicating that registration is unnecessary because the data has already been registered, but the types of marks are not limited to these. In other words, if it is determined that registration is unnecessary because the read data Dc is not a read target, a mark different from the marks Ma and Mb may be displayed as a mark indicating this determination result.

[0061] When the service unit 88 confirms a user's flick operation (registration instruction operation) on the display 20 displaying the image view window 93 (step S105), it transmits a registration instruction to the user application 87 to register the read data Dc5 and Dc6 determined to require registration in the read result 952 of the data management table 95 (step S106). Upon receiving the registration instruction from the service unit 88, the user application 87 registers the read data Dc5 and Dc6 indicated by the registration instruction in the read result 952 of the corresponding read targets Tx5 and Tx6 (the "Registration (second time)" column in FIG. 5C).

[0062] FIG. 6A is a flowchart showing an example of code reading performed by a mobile information terminal, FIG. 6B is a flowchart showing an example of a single code reading mode performed in the code reading of FIG. 6A, FIG. 6C is a flowchart showing an example of a multiple code reading mode performed in the operation of FIG. 6A, and FIG. 6D is a flowchart showing an example of a data saving process performed in the code reading of FIG. 6A.

[0063] 6A, the main body of the mobile information terminal 1 is activated, and the display 20 starts up. In step S302, the camera control unit 54 reads out code reading setting values ​​from the nonvolatile storage unit 62 and sets them in the camera module 3. In step S302, the camera control unit 54 also initializes the ISP 731 and the like. In step S303, the camera control unit 54 puts the camera module 3 to sleep.

[0064] Then, when the camera control private API 52 receives a code reading instruction from the code reading processing unit 72 ("YES" in step S304), the private API control unit 56 checks whether the single-code reading mode is set (step S305). That is, the user can set the private API control unit 56 to execute either the single-code reading mode or the multiple-code reading mode by operating the display 20. If the single-code reading mode is set ("YES" in step S305), the single-code reading mode of FIG. 6B is executed (step S306). On the other hand, if the multiple-code reading mode is set ("NO" in step S305), the multiple-code reading mode of FIG. 6C is executed (step S307).

[0065] 6B, the private API control unit 56 starts the single code reading mode. When the service unit 88 confirms that the trigger button 23 has been pressed (step S402), the private API control unit 56 activates the rear camera unit 32 of the camera module 3 (step S403). As described above, the image view window 93 is displayed on the display 20 in response to the activation of the rear camera unit 32.

[0066] Then, the private API control unit 56 generates a code image Ic using the rear camera unit 32 (step S404) and temporarily stores this code image Ic in the volatile storage unit 61 (step S405). In step S406, the decoder 86 performs a decoding process on the single code C included in the code image Ic, and read data Dc is acquired. As described above, the decoding process is executed when a predetermined waiting time (e.g., one second) has elapsed since it was confirmed that the trigger button 23 was pressed, or when it was confirmed that the trigger button 23 was pressed. In step S407, the service unit 88 transmits the read data Dc acquired from the decoder 86 to the user application 87. Then, this read data Dc is registered in the data management table 95 of the user application 87.

[0067] In step S501 of FIG. 6C, the private API control unit 56 starts the multiple code reading mode. In step S502, the private API control unit 56 activates the rear camera unit 32 of the camera module 3. As described above, the activation of the rear camera unit 32 causes the image view window 93 to be displayed on the display 20, and the service unit 88 displays the code image Ic captured by the rear camera unit 32 in the image view window 93 (step S503). At this time, the service unit 88 makes the area of ​​the image view window 93 displayed in the multiple code reading mode larger than the area of ​​the image view window 93 displayed in the single code reading mode. When multiple codes are included in the field of view, if the area of ​​the image view window 93 is constant, the size of each code becomes small. However, this configuration makes it possible to suppress a decrease in the size of the codes in the field of view.

[0068] When the service unit 88 confirms that the trigger button 23 has been pressed (step S504), it causes the decoder 86 to execute a decoding process on the code image Ic stored in the volatile storage unit 61 (step S505). Note that the code image Ic includes images of each of the multiple codes C, so the decoder 86 executes a decoding process on each of the multiple codes C to obtain multiple pieces of read data Dc. The service unit 88 transmits the multiple pieces of read data Dc obtained from the decoder 86 to the user application 87 (step S506), and the read data Dc is registered in the data management table 95 of the user application 87 via the flow described above with reference to Figures 5A and 5B.

[0069] 6A, when the single code reading mode (step S306) or the multiple code reading mode (step S307) is completed, the private API control unit 56 puts the camera module 3 into standby (step S308). Then, in step S309, the private API control unit 56 executes the data saving process of FIG. 6D (step S309).

[0070] 6D, the private API control unit 56 determines whether or not the code image Ic is set to be saved in the non-volatile storage unit 62. That is, the user can set whether or not to save the code image Ic in the non-volatile storage unit 62 by operating the display 20. If the saving in the non-volatile storage unit 62 is not set ("NO" in step S601), the private API control unit 56 deletes the code image Ic temporarily saved in the volatile storage unit 61 from the volatile storage unit 61 (step S604).

[0071] On the other hand, if storage in the non-volatile storage unit 62 is set (if "YES" in step S601), the private API control unit 56 determines whether the security mode is set (step S602). Specifically, the private API control unit 56 makes this determination based on the result of receiving a mode selection signal indicating whether the security mode or the normal mode is selected from the security setting unit 74. If the security mode is set (if "YES" in step S602), the private API control unit 56 deletes the code image Ic from the volatile storage unit 61 (step S604). On the other hand, if the security mode is not set (if "NO" in step S602), in other words, if the normal mode is set, the private API control unit 56 saves the code image Ic temporarily stored in the volatile storage unit 61 in the non-volatile storage unit 62 (step S603), and then deletes the code image Ic from the volatile storage unit 61 (step S604).

[0072] Note that execution of a screenshot is appropriately restricted for the display 20 displaying the image view window 93. In other words, when an instruction to execute a screenshot is received while the image view window 93 is displayed on the display 20 in the multiple code reading mode, the screenshot control unit 77 executes the flowchart of Fig. 6E. Here, Fig. 6E is a flowchart showing an example of screenshot control.

[0073] When the screenshot control unit 77 receives an instruction to execute a screenshot ("YES" in step S701), it determines whether or not the security mode is set (step S702). If the security mode is not set ("NO" in step S702), the screenshot control unit 77 saves a screenshot image of the display 20 displaying the image view window 93 in the non-volatile storage unit 62 (step S703).

[0074] On the other hand, if the security mode is set ("YES" in step S702), the screenshot prohibition unit 76 prohibits screenshots as described above. Therefore, the screenshot control unit 77 ignores the instruction to execute a screenshot and does not execute a screenshot (step S704). Note that in step S704, instead of prohibiting screenshots, the screenshot control unit 77 may invalidate (replace all pixel values ​​with a fixed pixel value) or mask (replace with a mask consisting of fixed pixel values) the screenshot image on the display 20 and store the image in the non-volatile storage unit 62.

[0075] In the mobile information terminal 1 described above, the conditions for executing code reading are controlled according to the distance to the workpiece W measured by the distance measuring sensor 37. Fig. 7A is a flowchart showing an example of the code reading condition control, and Figs. 7B, 7C, and 7D are diagrams schematically showing the operations executed in the code reading condition control of Fig. 7A.

[0076] In step S801, the service unit 88 causes the distance measurement sensor 37 to measure the distance to the workpiece W and acquires the distance measured by the distance measurement sensor 37. In step S802, the service unit 88 determines the position and size of the target range J to be displayed in the image view window 93 in the code image Ic acquired by the rear camera 33 capturing the camera field of view V33, according to the measured distance. This is to accommodate the fact that the distance between the center M33 of the camera field of view V33 (in other words, the center of the code image Ic) and the aimer light L36 and the size of the code C change in the code image Ic according to the distance to the workpiece W.

[0077] 7B shows a case where the distance to the workpiece W is relatively short, and FIG. 7C shows a case where the distance to the workpiece W is relatively long. As can be seen from these comparisons, when the distance to the workpiece W is short (FIG. 7B), the distance Δn between the center M33 of the camera field of view V33 and the aimer light L36 that appears in the code image Ic is wider than the distance Δf between the center M33 of the camera field of view V33 and the aimer light L36 that appears in the code image Ic when the distance to the workpiece W is long (FIG. 7C). In addition, the size of the code C that appears in the code image Ic when the distance to the workpiece W is short (FIG. 7B) is larger than the size of the code C that appears in the code image Ic when the distance to the workpiece W is long (FIG. 7C).

[0078] Therefore, the service unit 88 shifts the center Mj of the target range J of the code image Ic to be displayed in the image view window 93 toward the irradiation position of the aimer light L36, rather than the center M33 of the camera field of view V33. For example, the service unit 88 sets the center Mj of the target range J between the center M33 of the camera field of view V33 and the irradiation position of the aimer light L36. In particular, the service unit 88 narrows the interval R between the center Mj of the target range J in the camera field of view V33 and the center M33 of the camera field of view V33, as the measurement distance increases. Furthermore, the service unit 88 reduces the size of the target range J in the camera field of view V33 as the measurement distance increases. As a result, the aimer light L36 and the code C become larger as the measurement distance increases. Then, in step S803 (FIG. 7A), the service unit 88 displays the target range J of the code image Ic determined in step S802 in the image view window 93.

[0079] In step S804, the service unit 88 calculates depth of field information 96 based on the measured distance acquired in step S801 (FIG. 7D). The depth of field information 96 includes position coordinates 961 up to infinity, a depth of field 962 at the position coordinates 961, and a focus position 963 of the rear camera 33. The depth of field 962 is calculated based on the measured distance. As described above, the focus position 963 is adjusted to match the position of the workpiece W indicated by the measured distance. The service unit 88 displays the depth of field information 96 shown in FIG. 7D in the image view window 93. Furthermore, the service unit 88 compares the depth of field 962 with a predetermined depth threshold, and displays a warning in the image view window 93 if the depth of field 962 is less than the depth threshold.

[0080] In step S805, the service unit 88 transmits the measured distance to the decoder 86, and the decoder 86 calculates a binning size according to the measured distance. Specifically, the decoder 86 calculates the binning size according to the measured distance so that the binning size increases as the measured distance decreases. Here, binning is an image process that combines adjacent pixels in the code image Ic into one pixel of a predetermined binning size, and the decoder 86 performs a decoding process on the code image Ic that has been binned.

[0081] Then, steps S801 to S805 are repeatedly executed until the display of the image view window 93 is completed ("YES" in step S806).

[0082] Incidentally, an attachment 4 can be detachably attached to the housing 11 of the mobile information terminal 1. Figures 8A to 8E are rear views showing an example of an attachment, showing the attachment 4 attached to the housing 11. The attachment 4 has an attachment cover 41 attached to the back surface 13 of the housing 11, and when attached to the housing 11, the attachment cover 41 overlaps the rear side of the rear camera unit 32 in the thickness direction Xt.

[0083] Windows 413, 414, 415, 416, and 417 are opened in this attachment cover 41. When the attachment 4 is attached to the housing 11, the window 413 faces the rear camera 33, the window 414 faces the wide-angle illumination unit 34, the window 415 faces the narrow-angle illumination unit 35, the window 416 faces the aimer light illumination unit 36, and the window 417 faces the distance measurement sensor 37. Note that a pair of windows 417 are provided for the distance measurement sensor 37, corresponding to an optical system that emits distance measurement light L37 and an optical system that collects the distance measurement light L37 on the optical sensor. Therefore, the rear camera 33 can capture the camera field of view V33 through the window 413, the wide-angle illumination unit 34 can emit wide-angle illumination light L34 through the window 414, the narrow-angle illumination unit 35 can irradiate narrow-angle illumination light L35 through the window 415, the aimer light illumination unit 36 ​​can irradiate aimer light L36 through the window 416, and the distance measurement sensor 37 can irradiate and detect distance measurement light L37 through the window 417.

[0084] 8A, a polarizing plate 423 is attached to a window 413 for the rear camera 33, and a polarizing plate 425 is attached to a window 415 for the narrow-angle illumination unit 35, and the polarization angles of the polarizing plate 423 and the polarizing plate 425 are orthogonal to each other. Therefore, narrow-angle illumination light L35 emitted from the narrow-angle illumination unit 35 is polarized by the polarizing plate 425 and then irradiated onto a camera field of view V33 of the rear camera 33. Furthermore, the rear camera 33 captures an image of the camera field of view V33 using the light polarized by the polarizing plate 423.

[0085] 8B, a light diffusion plate 43 is attached to the window 414 for the wide-angle illumination unit 34. Therefore, the wide-angle illumination light L34 emitted from the wide-angle illumination unit 34 is diffused by the light diffusion plate 43 and then irradiated onto the camera field of view V33 of the rear camera 33. This wide-angle illumination light L34 functions as a dome-shaped illumination light that irradiates the camera field of view V33 from all directions.

[0086] 8C, a prism mirror 44 is attached to the window 414 for the wide-angle illumination unit 34. Therefore, the wide-angle illumination light L34 emitted from the wide-angle illumination unit 34 is irradiated onto the camera field of view V33 of the rear camera 33 via the prism mirror 44. This prism mirror 44 has the function of reducing the angle at which the wide-angle illumination light L34 is irradiated onto the camera field of view V33, in other words, has the function of irradiating the wide-angle illumination light L34 onto the camera field of view V33 as low-angle illumination.

[0087] 8D, a color filter 45 is attached to the window 414 for the wide-angle illumination unit 34. Therefore, the wide-angle illumination light L34 emitted from the wide-angle illumination unit 34 is irradiated onto the camera field of view V33 of the rear camera 33 via the color filter 45. This color filter 45 has a function of transmitting only light of a specific color (specific wavelength) out of the wide-angle illumination light L34.

[0088] 8A to 8E, for example, four types of attachments 4 can be attached to and detached from the housing 11. In response to this, the mobile information terminal 1 can determine the type of attachment 4 attached to the housing 11. Fig. 9 is a block diagram showing the electrical configuration of a mobile information terminal that determines the type of attachment.

[0089] 9, the mobile information terminal 1 has a short-range wireless communication unit 38 and an activation control unit 39. An identifier that identifies the type of attachment 4 is built into an attachment cover 41 of the attachment 4, and the short-range wireless communication unit 38 reads the identifier of the attachment 4 by NFC (Near Field Communication) to determine the type of attachment 4. The activation control unit 39 also controls activation of the short-range wireless communication unit 38 in accordance with the distance measured by the distance measurement sensor 37.

[0090] That is, the activation control unit 39 detects the approach of the attachment 4 to the housing 11 based on the distance measured by the distance measurement sensor 37. If the distance measured by the distance measurement sensor 37 is longer than the communication distance by NFC, the activation control unit 39 puts the short-range wireless communication unit 38 to sleep. On the other hand, if the measurement distance of the distance measurement sensor 37 becomes equal to or shorter than the communication distance by NFC, the activation control unit 39 determines that the attachment 4 has approached the housing 11, and activates the short-range wireless communication unit 38. Then, the activated short-range wireless communication unit 38 reads the identifier of the attachment 4 and determines the type of the attachment 4.

[0091] In the present embodiment described above, the user application 87 determines whether or not the read data Dc, which is the result of the decoding process received from the service unit 88, needs to be registered in the user application 87, and transmits the determination result on whether or not the registration is needed to the service unit 88 (steps S102 to S103 in FIG. 5A). Then, the service unit 88 displays on the display 20 an image view window 93 showing marks Ma and Mb (information) indicating the determination result received from the user application 87 superimposed on the code image Ic (FIG. 5C). Therefore, the user can easily check whether or not the read data Dc (image processing result) needs to be registered, based on the marks Ma and Mb displayed in the image view window 93. In this way, the user can easily check whether or not the result of the decoding process using the rear camera 33 of the mobile information terminal 1 needs to be registered in the user application 87.

[0092] Furthermore, the user application 87 has data registration conditions 951 including at least one condition regarding the read data Dc that needs to be registered in the user application 87, and determines whether or not registration is necessary based on the data registration conditions 951 (step S202 in FIG. 5B, FIG. 5C). With this configuration, whether or not registration of the read data Dc is necessary can be easily and automatically determined based on the data registration conditions 951.

[0093] The data registration conditions 951 also include conditions related to any of the type of code C to be decoded, the character string included in code C to be decoded, the numerical value included in code C to be decoded, and the position of code C to be decoded. With this configuration, it is possible to easily and automatically determine whether or not the read data Dc resulting from the decoding process needs to be registered, using appropriate conditions according to the code C to be decoded.

[0094] Furthermore, based on the determination result received from the user application 87, the service unit 88 causes the display 20 to display an image view window 93 in which the code C that is the target of the decoding process and is included in the code image Ic being displayed in the image view window 93 and that needs to be registered in the user application 87 is identified by a mark Ma (step S104 in FIG. 5A, FIG. 5C). With this configuration, the user can easily confirm from the image view window 93 that the read data Dc of the identified code C needs to be registered.

[0095] Furthermore, the service unit 88 displays an image view window 93 on the display 20, which distinguishes between codes C that need to be registered, codes C that have already been registered and do not need to be registered, and codes C that have not yet been registered and do not need to be registered, among the codes C that are the subject of image processing. With this configuration, the user can easily check the registration status and necessity of each code C that is the subject of decoding processing based on the image view window 93.

[0096] Furthermore, when the decoder 86 receives a flick operation (registration operation) from the user while the image view window 93, in which the code C that needs to be registered is identified by the mark Ma, is displayed on the display 20, the decoder 86 registers the read data Dc of the code C that needs to be registered (steps S105 to S106 in FIG. 5A). In this configuration, the user can easily register the read data Dc of the code C that needs to be registered in the user application 87 by performing a flick operation.

[0097] Furthermore, this embodiment can address the following problem. That is, there is a need for a method for easily reading codes using a mobile information terminal, such as a smartphone. However, the cameras and lighting installed in such mobile information terminals are designed for taking photos of people or landscapes, and are not necessarily suitable for reading codes. As a result, focusing and exposure require time, making it difficult to quickly read codes.

[0098] In contrast, this embodiment enables the code to be read quickly using the camera of the portable information terminal used to take the photograph.

[0099] The portable information terminal according to this embodiment includes a housing having a first main surface on which a display unit is provided and a second main surface different from the first main surface, a camera with a variable focus position that captures a field of view and generates a captured image, a wide-angle illumination unit that irradiates the field of view of the camera with wide-angle illumination light, a narrow-angle illumination unit that irradiates the field of view of the camera with narrow-angle illumination light that is narrower than the wide-angle illumination light, an aimer light irradiation unit that irradiates the work with aimer light, which is visible light, a distance measurement sensor that irradiates distance measurement light on the work and measures the distance to the work based on the result of detecting the distance measurement light reflected by the work, a focus adjustment unit that adjusts the focus position of the camera according to the distance measured by the distance measurement sensor, and a focus adjustment unit that adjusts the focus position of the camera based on the captured image of the work. The camera includes a decoder that decodes the code attached to the workpiece, and an imaging control unit that controls the conditions under which the camera takes an image. The camera, wide-angle illumination unit, narrow-angle illumination unit, aimer light irradiation unit, and distance measurement sensor are provided on the second main surface of the housing. When taking a photograph, the imaging control unit causes the wide-angle illumination unit to irradiate wide-angle illumination light while the camera is taking an image, thereby causing the camera to generate an image for the photograph. When decoding, the imaging control unit causes the aimer light irradiation unit to irradiate the workpiece with aimer light, and the narrow-angle illumination unit to irradiate narrow-angle illumination light while the focus adjustment unit adjusts the focus position of the camera, thereby causing the camera to take an image and generate an image for the decoding process.

[0100] In this embodiment, a housing 11 having a front surface 12 (first main surface) and a back surface 13 (second main surface) is provided, a display 20 (display unit) is provided on the front surface 12, and a rear camera 33 and a wide-angle illumination unit 34 are provided on the back surface 13. The wide-angle illumination unit 34 irradiates a camera field of view V33 of the rear camera 33 with wide-angle illumination light L34, and the rear camera 33 captures the camera field of view V33, thereby generating a photographic image Ip (captured image) (photography). Furthermore, a narrow-angle illumination unit 35, an aimer light illumination unit 36, and a distance measurement sensor 37 are provided on the back surface 13. The narrow-angle illumination unit 35 irradiates the camera field of view V33 of the rear camera 33 with narrow-angle illumination light L35, which has a narrower light distribution angle than the wide-angle illumination light L34. The aimer illumination unit 36 ​​irradiates the workpiece W with aimer light L36, which is visible light. The distance measurement sensor 37 measures the distance to the workpiece W based on the result of irradiating the workpiece W with the aimer light L37 and detecting the distance measurement light L37 reflected by the workpiece W. Then, in the decoding process, the following operation is executed. That is, the user holds the mobile information terminal 1 over the workpiece W so that the aimer light L36 is irradiated from the aimer light illumination unit 36 ​​onto the workpiece W. As a result, the distance measurement light L37 from the distance measurement sensor 37 and the narrow-angle illumination light L35 from the narrow-angle illumination unit 35 are irradiated onto the workpiece W. As a result, the focus position of the rear camera 33 is quickly adjusted according to the distance measured by the distance measurement sensor 37, and the workpiece W is imaged with the narrow-angle illumination light L35 by a relatively short exposure, thereby generating a code image Ic. The decoder 86 then decodes the code C based on the code image Ic. In this way, the code C can be quickly read using the rear camera 33 of the mobile information terminal 1 used for taking photos. Note that the rolling shutter method is more susceptible to camera shake than the global shutter method. To reduce the effects of camera shake, it is necessary to shorten the exposure time of the rear camera 33. Here, because the narrow-angle illumination light L35 has a higher illuminance than the wide-angle illumination light L34, the use of the narrow-angle illumination light L35 can shorten the exposure time.

[0101] Furthermore, the aimer light L36 and the distance measurement light L37 are included in the camera field of view V33 of the rear camera 33 and are also included in the range irradiated by the narrow-angle illumination light L35, and the irradiation positions of the aimer light L36 and the distance measurement light L37 on the workpiece W are included in the depth of field of the rear camera 33, the focus position of which has been adjusted by the non-public API control unit 56 (focus adjustment unit). With this configuration, as the user holds the mobile information terminal 1 over the workpiece W so that the aimer light L36 is irradiated from the aimer light illuminator 36 onto the workpiece W, the distance measurement light L37 is irradiated within the irradiation range of the narrow-angle illumination light L35 on the workpiece W, and the focus of the rear camera 33 can be adjusted to this irradiation range based on the distance measured by the distance measurement sensor 37. As a result, the code image Ic can be generated by the rear camera 33 with the code C irradiated with the narrow-angle illumination light L35 in focus.

[0102] Furthermore, the optical axes of the aimer light L36, the distance measurement light L37, the camera field of view V33 of the rear camera 33, and the narrow-angle illumination light L35 are parallel to one another. With this configuration, the positional relationship between the aimer light L36, the distance measurement light L37, the camera field of view V33 of the rear camera 33, and the narrow-angle illumination light L35 can be kept uniform regardless of the distance to the workpiece W.

[0103] Furthermore, the distance K35 between the optical axis of the narrow-angle illumination light L35 and the rear camera 33 is equal to or less than the distance K34 between the optical axis of the wide-angle illumination unit 34 and the optical axis of the rear camera 33. In this configuration, the narrow-angle illumination unit 35 is disposed close to the rear camera 33, and the narrow-angle illumination light L35 can be accurately irradiated onto the code C within the camera field of view V33 of the rear camera 33.

[0104] The rear camera 33 is arranged in a line between any two of the narrow-angle illumination unit 35, the aimer light illumination unit 36, and the distance measurement sensor 37 (FIG. 1E, FIGS. 11A to 11D). In this configuration, the narrow-angle illumination light L35, the aimer light L36, and the distance measurement light L37 can be placed close to each other and contained within the camera field of view V33 of the rear camera 33.

[0105] Also provided is an attachment 4 for the rear camera 33 that is detachably attached to the housing 11. The attachment 4 has a polarizing plate 423 (optical component) that faces the rear camera 33 when attached to the housing 11 and adjusts the characteristics of light that enters the rear camera 33. With this configuration, an appropriate polarizing plate 423 according to the type of cord C, etc., can be used to capture an image with the rear camera 33.

[0106] Also provided is an attachment 4 for the narrow-angle illumination unit 35 that is detachably attached to the housing 11. The attachment 4 has a polarizing plate 425 (optical component) that faces the narrow-angle illumination unit 35 and adjusts the characteristics of the narrow-angle illumination light L35 emitted from the narrow-angle illumination unit 35 when attached to the housing 11. With this configuration, the narrow-angle illumination light L35 can be emitted using an appropriate polarizing plate 425 depending on the type of cord C, etc.

[0107] The device also includes a short-range wireless communication unit 38 (communication unit) that performs short-range wireless communication, and a startup control unit 39 that controls sleep and startup of the short-range wireless communication unit 38. The attachment 4 is provided with an identifier that identifies the attachment 4, and the startup control unit 39 determines whether the attachment 4 is attached to the housing 11 based on the distance detected by the distance measurement sensor 37. The startup control unit 39 puts the short-range wireless communication unit 38 to sleep when it determines that the attachment 4 is not attached, but starts the short-range wireless communication unit 38 when it determines that the attachment 4 is attached. When the short-range wireless communication unit 38 starts up, the identifier of the attachment 4 is recognized by the short-range wireless communication unit 38. With this configuration, the identifier of the attachment 4 attached to the housing 11 can be recognized by short-range wireless communication of the short-range wireless communication unit 38. Moreover, based on the distance detected by the distance measurement sensor 37, the short-range wireless communication unit 38 can be started to recognize the identifier only when the attachment 4 is attached to the housing 11, thereby reducing power consumption in the short-range wireless communication unit 38.

[0108] Furthermore, the decoder 86 changes the execution mode of the decoding process for the code image Ic in accordance with the distance detected by the distance measurement sensor 37 (FIG. 7A). This allows the decoding process for the code image Ic to be executed in an appropriate execution mode in accordance with the distance to the workpiece W.

[0109] For example, the decoder 86 performs a decoding process on the result of performing binning on the code image Ic, which combines adjacent pixels into one pixel of a predetermined size. At this time, the decoder 86 changes the binning size according to the distance detected by the distance measurement sensor 37 (step S806 in FIG. 7A). With this configuration, the code image Ic can be binned at an appropriate size according to the distance to the workpiece W.

[0110] Furthermore, the decoder 86 executes decoding processing on a target range J of the code image Ic, which has a center Mj (target position) that is biased toward a position corresponding to the irradiation position of the aimer light L36 from a center M33 (imaging center) that is the center of the code image Ic. At this time, the service unit 88 changes the target range J according to the distance detected by the distance measuring sensor 37 (step S802 in FIG. 7A). With this configuration, it is possible to execute decoding processing by narrowing it down to the range biased toward the irradiation position of the aimer light L36, in other words, the range in which the image of the code image Ic exists.

[0111] Furthermore, the decoder 86 executes a decoding process on a target range J of the code image Ic. At this time, the service unit 88 changes the size of the target range J according to the distance detected by the distance measurement sensor 37 (step S802 in FIG. 7A). With this configuration, the decoding process can be executed by narrowing down the target range J according to the distance to the workpiece W, in other words, the range in which the code image Ic exists.

[0112] The housing 11 is also provided with a trigger button 23 that generates a trigger for causing the service unit 88 (imaging control unit) to start generating a code image Ic using the rear camera 33 and executing a decoding process for the code image Ic. The housing 11 has a shape that extends in the longitudinal direction Xl, and has a right side surface 14 (side surface) that connects the front surface 12 (first main surface) and the back surface 13 (second main surface) and extends along the longitudinal direction Xl. The right side surface 14 has a recess 141 that is recessed inward between the front surface 12 and the back surface 13. The trigger button 23 has a base portion 231 that is disposed within the recess 141 and extends in the longitudinal direction, and a protrusion portion 232 that protrudes from the base portion 231 to the outside of the right side surface 14. With this configuration, the user can visually or tactilely recognize the protrusion portion 232 of the trigger button 23 that protrudes from the recess 141, thereby enabling the user to reliably operate the trigger button 23.

[0113] Furthermore, the trigger button 23 is positioned offset upward from the center M11 of the housing 11 in the longitudinal direction (FIG. 1C). With this configuration, the trigger button 23 is offset from the center M11 of the housing 11, which is easy for the user to grip, and this prevents the user from accidentally operating the trigger button 23 when picking up the mobile information terminal 1. Furthermore, since the user's thumb and index finger tend to be positioned on the upper side of the housing 11 when reading, the trigger button 23 is easy to operate even when the mobile information terminal 1 is held over a code C in a location that is difficult to read.

[0114] Furthermore, this embodiment can address the following issue. That is, there is a need for a portable information terminal, such as a smartphone, to easily read the above-mentioned codes and perform image processing such as OCR (Optical Character Recognition / Reader). This need can be met by using a camera mounted on the portable information terminal to capture an image of an object to be processed. However, this camera can also be used to take photos of people and landscapes, and images captured by the camera may contain information other than the object to be processed. For this reason, in areas where confidential information is handled, it has been difficult to use a camera mounted on such a portable information terminal to capture an image of an object to be processed.

[0115] In contrast to this, this embodiment makes it possible to prevent images captured by a camera mounted on a portable information terminal from leaking to the outside when the camera is used to capture an image to be processed.

[0116] The portable information terminal of this embodiment comprises a camera that captures an image of a workpiece and generates the captured image; a reading unit that performs image processing based on the captured image to acquire information contained in the workpiece; a volatile first memory unit that temporarily stores the captured image; a non-volatile second memory unit that stores image processing settings and is capable of storing the captured image; a control unit that causes the camera to generate the captured image and store it in the first memory unit and causes the reading unit to perform image processing based on the stored captured image; and a mode selection unit that selects either a first mode in which the captured image generated by the camera is stored in the second memory unit, or a second mode in which the captured image generated by the camera is deleted from the first memory unit without being stored in the second memory unit, or the captured image stored in the second memory unit is forcibly deleted from the second memory unit.

[0117] The captured image management method of this embodiment is a captured image management method that manages the storage of captured images generated by a camera by capturing an image of a workpiece in a first memory unit or a second memory unit, and includes the steps of selecting either a first mode in which the captured image generated by the camera is stored in the second memory unit, or a second mode in which the captured image generated by the camera is deleted from the first memory unit without being stored in the second memory unit, or the captured image stored in the second memory unit is forcibly deleted from the second memory unit, and having the camera generate the captured image and store it in the first memory unit, and performing image processing based on the stored captured image to obtain information contained in the workpiece, wherein the first memory unit is a volatile memory unit that temporarily stores the captured image, and the second memory unit is a non-volatile memory unit that stores image processing settings and is capable of storing the captured image.

[0118] In this embodiment, a volatile storage unit 61 (first storage unit) and a non-volatile storage unit 62 (second storage unit) are provided as storage units capable of storing captured images (photographic image Ip, code image Ic) (FIG. 2A). The volatile storage unit 61 is used for the decoding process. That is, the code image Ic generated by the rear camera 33 is stored in the volatile storage unit 61, and the decoder 86 (reading unit) performs the decoding process based on the stored code image Ic. In this manner, the code C (image processing target) is captured using the rear camera 33 of the mobile information terminal 1, and the decoding process is performed. In addition, a normal mode (first mode) and a security mode (second mode) are provided as modes for storing the code image Ic generated by the rear camera 33. In the normal mode, the code image Ic generated by the rear camera 33 is stored in the non-volatile storage unit 62 (step S603 in FIG. 6D). In the security mode, the code image Ic generated by the rear camera 33 is deleted from the volatile storage unit 61 without being stored in the nonvolatile storage unit 62 (step S604 in FIG. 6D). Note that in the security mode, instead of this operation, a step of forcibly deleting the code image Ic stored in the nonvolatile storage unit 62 from the nonvolatile storage unit 62 may be executed. In this configuration, by selecting the security mode, the code image Ic generated by the rear camera 33 is deleted, and therefore, is not leaked to the outside. In this way, when the rear camera 33 mounted on the mobile information terminal 1 is used to capture an image of the code C to be decoded, it is possible to prevent the code image Ic captured by the rear camera 33 from being leaked to the outside.

[0119] The mobile information terminal 1 further includes a display 20, and the service unit 88 (control unit) displays (image view window 93) on the display 20 the code image Ic that is stored in the volatile storage unit 61 and is to be decoded by the decoder 86, whether the normal mode or the security mode is selected by the security setting unit 74 (mode selection unit). With this configuration, the user operating the mobile information terminal 1 can perform work while visually checking the code image Ic that is to be decoded on the display 20, improving user operability.

[0120] Furthermore, when the security setting unit 74 selects the security mode, the screenshot prohibition unit 76 (control unit) prohibits taking a screenshot of the display 20 displaying the code image Ic, or stores an invalid or masked image of the screenshot of the display 20 displaying the code image Ic in the non-volatile storage unit 62 (FIG. 6E). With this configuration, it is possible to prevent the image of the screenshot from being leaked to the outside.

[0121] The rear camera 33 is also provided with a photography processing unit 71 that generates a photographic image Ip using the rear camera 33 and stores the photographic image Ip in the non-volatile storage unit 62. The control unit 5 controls the rear camera 33 via either a public camera control API 51 (public API) that controls the rear camera 33 to generate a photographic image Ip in response to a request from the photography processing unit 71, or a private camera control API 52 (private API) that controls the rear camera 33 to generate a code image Ic in response to a request from a code reading processing unit 72 (reading unit). That is, in normal mode, control of the rear camera 33 via both the public camera control API 51 and the private camera control API 52 is permitted. In security mode, control of the rear camera 33 via the private camera control API 52 is permitted, while control of the rear camera 33 via the public camera control API 51 is prohibited. In this configuration, separate APIs (public API and private API) are used for photography and decoding, and by using these APIs appropriately, security for photos taken by the rear camera 33 can be easily switched.

[0122] Also provided is a telephone communication unit 83 that communicates via a telephone line. When the security setting unit 74 selects a security mode, the communication control unit 81 (control unit) permits voice calls via the telephone communication unit 83 with telephone terminals with registered telephone numbers, but prohibits voice calls via the telephone communication unit 83 with telephone terminals with unregistered telephone numbers. With this configuration, it is possible to limit the range of telephone use and prevent information from leaking to the outside.

[0123] Furthermore, when the security setting unit 74 selects the security mode, the communication control unit 81 prohibits the telephone communication unit 83 from recording voice calls. With this configuration, it is possible to prevent recordings of voice calls from being leaked to the outside.

[0124] The device is also provided with an Internet communication unit 82 that communicates via the Internet. When the security setting unit 74 selects a security mode, the communication control unit 81 prohibits the Internet communication unit 82 from connecting to the Internet. With this configuration, it is possible to prevent information from leaking to the outside via the Internet.

[0125] Furthermore, this embodiment can address the following issue. That is, there is a need for a portable information terminal, such as a smartphone, to easily read the above-mentioned codes and perform image processing such as OCR (Optical Character Recognition / Reader). This need can be met by using a camera mounted on the portable information terminal to capture an image of an object to be processed. In particular, by displaying the field of view of the camera on the display unit (display) of the smartphone, the user can perform operations while confirming that the object to be processed is captured in the field of view of the camera.

[0126] However, displaying Gamera's field of view on the display unit can impair user operability. That is, a user application screen used for user operation is displayed on the camera's display unit. This user application screen has predetermined input fields, and the result of image processing is input into the input field focused on (in other words, selected) by the user. In this case, as the camera's field of view is displayed, the display on the display unit switches from the user application screen to the camera's field of view, which causes the user to lose track of the input field they are operating.

[0127] In contrast to this, the present embodiment makes it possible to display the field of view of the camera while ensuring user operability of the user application screen.

[0128] The mobile information terminal of this embodiment comprises a camera that captures an image of a workpiece and generates the captured image; an image processing unit that performs image processing based on the captured image; a service unit that transmits the results of the image processing to a user application that has at least one input item and reflects the results of the image processing in the input item; and a display unit that displays a user application screen when the user application is executed.When the camera is started to perform image processing during execution of the user application, the service unit displays an image view window on the display unit that shows at least a portion of the camera's field of view together with the user application screen, while keeping the input item that is focused as the input target among the input items on the user application screen visible to the user.

[0129] The mobile information terminal control method of this embodiment includes the steps of capturing an image of a workpiece using a camera mounted on the mobile information terminal to generate an captured image, performing image processing based on the captured image, transmitting the results of the image processing by a service unit to a user application that has at least one input item and reflects the results of the image processing in the input item, and displaying a user application screen on a display unit when the user application is executed.When the camera is activated to perform image processing during execution of the user application, the service unit causes an image view window showing at least a portion of the camera's field of view together with the user application screen to be displayed on the display unit, while keeping the input item that is focused as the input target among the input items on the user application screen visible to the user.

[0130] In this embodiment, when the rear camera 33 (camera) is activated to execute the decoding process (image processing), an image view window 93 showing at least a part of the camera field of view V33 of the rear camera 33 is displayed on the display 20 (display unit) together with the user application screen 91, while keeping the input item focused as the input target among the input items on the user application screen 91 visible to the user (FIG. 3A). In this way, it is possible to display the camera field of view V33 of the rear camera 33 while ensuring user operability of the user application screen 91.

[0131] Furthermore, the service unit 88 displays the image view window 93 semi-transparently over the user application screen 91, and the area of ​​the user application screen 91 that overlaps with the image view window 93 is displayed semi-transparently through the image view window 93. In this configuration, the user can view the user application screen 91 displayed semi-transparently through the image view window 93. In this way, it is possible to display the camera field of view V33 of the rear camera 33 while ensuring user operability of the user application screen 91.

[0132] Furthermore, the service unit 88 increases the transparency of the image view window 93 as the saturation of the image displayed in the image view window 93 increases. In this configuration, the transparency of the image view window 93 (the degree to which the user application screen 91 is transparent to the image view window 93) can be adjusted appropriately according to the saturation of the image view window 93. As a result, it is possible to display the camera field of view V33 of the rear camera 33 while ensuring user operability of the user application screen 91.

[0133] The system is also provided with a trigger button 23 that generates a trigger to cause the decoder 86 (image processing unit) to execute decoding processing (image processing). When the trigger button 23 is operated, the service unit 88 displays an image view window 93 on the display 20, and closes the image view window 93 on the display 20 after the decoding processing is completed. With this configuration, the image view window 93 can be displayed at a timing corresponding to the operation of the trigger button 23.

[0134] Furthermore, the service unit 88 waits a predetermined period of time after the trigger button 23 is operated, and then causes the decoder 86 to execute the decoding process, and when the decoding process is complete, closes the image view window 93 on the display 20. In this configuration, the decoding process is executed after waiting a predetermined period of time after the trigger button 23 is operated. As a result, the image view window 93 does not close immediately upon completion of the decoding process before the user has time to check the code C to be decoded, and the user can reliably check the code C to be decoded in the image view window 93.

[0135] The decoder 86 is also capable of executing step S505 (multiple code processing) in FIG. 6C, which performs decoding processing on each of the multiple codes C attached to the work W, and step S406 (single code processing) in FIG. 6B, which performs decoding processing on one code C attached to the work W. In response to this, the decoder 86 makes the size of the image view window 93 displayed on the display 20 in step S505 (multiple code processing) larger than the size of the image view window 93 displayed on the display 20 in step S406 (single code processing). With this configuration, the image view window 93 can be displayed in an appropriate size according to the number of codes C to be decoded.

[0136] Furthermore, the service unit 88 displays a portion (target range J) of the code image Ic (captured image) in the image view window 93. With this configuration, the portion (target range J) of the code image Ic can be displayed large in the image view window 93. The service unit 88 displays only a portion of the field of view V33 of the rear camera 33 in the image view window 93. The service unit 88 excludes areas of the captured image other than the portion of the field of view V33 of the rear camera 33 from the target of image processing. This is to avoid confusion that may occur if a code not visible in the image view window 93 is read, resulting in the user not knowing which code has been read. As another example, the image sensor of the rear camera 33 partially outputs pixel signals from a portion of the area, and the service unit 88 displays the captured image corresponding to the portion of the image sensor in the image view window 93. This narrows the readout area from the image sensor, thereby speeding up processing.

[0137] The system is also provided with a distance measurement sensor 37 that measures the distance to the workpiece W based on the results of irradiating distance measurement light L37 onto the workpiece W and detecting the distance measurement light L37 reflected by the workpiece W. The service unit 88 displays a target range J of the code image Ic in an image view window 93, and changes the size of the target range J according to the distance detected by the distance measurement sensor 37 (steps S801 and S802 in FIG. 7A). With this configuration, an appropriate target range J according to the distance to the workpiece W can be displayed in the image view window 93.

[0138] The system also includes an aimer light illumination unit 36 ​​that irradiates the workpiece W with an aimer light L36, which is visible light, and a distance measurement sensor 37 that irradiates the workpiece W with a distance measurement light L37 and measures the distance to the workpiece W based on the result of detecting the distance measurement light L37 reflected by the workpiece W. The service unit 88 displays, in an image view window 93, a target range J centered on a center Mj (target position) of the code image Ic that is biased from the center M33 (imaging center) of the code image Ic to a position corresponding to the irradiation position of the aimer light L36. At this time, the service unit 88 changes the center Mj of the target range J according to the distance detected by the distance measurement sensor 37 (FIGS. 7B and 7C). With this configuration, decoding processing can be performed by narrowing the range biased to the irradiation position of the aimer light L36, in other words, the range in which the image of the code C to be decoded exists.

[0139] The system is also provided with a distance measurement sensor 37 that irradiates distance measurement light L37 onto the workpiece W and measures the distance to the workpiece W based on the results of detecting the distance measurement light L37 reflected by the workpiece W. The service unit 88 calculates the depth of field of the rear camera 33 from the distance to the workpiece W measured by the distance measurement sensor 37, and displays it in the image view window 93 (step S804 in FIG. 7A, FIG. 7D). With this configuration, the user can check the appropriateness of the depth of field through the image view window 93.

[0140] Furthermore, if the depth of field is less than the threshold, the service unit 88 displays a warning in the image view window 93. In this configuration, the user can confirm through the image view window 93 that the depth of field is inappropriate.

[0141] The present invention is not limited to the above embodiment, and various modifications can be made to the above without departing from the spirit of the present invention. For example, the security mode setting method may be modified as shown in FIG. 10. FIG. 10 is a flowchart showing a modified example of the security mode setting method. In the modified example of FIG. 10, the security setting unit 74 (FIG. 2A) determines whether the administrator has performed an operation on the administrator setting screen to instruct setting of the security mode (step S901). If there is no instruction to set the security mode ("NO" in step S901), the security setting unit 74 sets the normal mode in step S904 (i.e., does not set the security mode).

[0142] On the other hand, if there is an instruction to set the security mode (if "YES" in step S901), the security setting unit 74 determines whether the mobile information terminal 1 is within a predetermined security area (step S902). Specifically, the security setting unit 74 acquires the position information of the mobile information terminal 1 calculated by the position information calculation unit 78, and determines whether the position of the mobile information terminal 1 indicated by the position information is included in the security area. Then, if the position of the mobile information terminal 1 is not included in the security area (if "NO" in step S902), the security setting unit 74 sets the normal mode in step S904. On the other hand, if the position of the mobile information terminal 1 is included in the security area (if "YES" in step S902), the security setting unit 74 sets the security mode in step S903.

[0143] That is, in this modification, the mode selection signal generated by the security setting unit 74 indicates whether to select normal mode or security mode based on the setting by the administrator authority and the location information of the mobile information terminal 1. With this configuration, for example, it is possible to select security mode when the location information indicates a location within a security area, and select normal mode when the location information indicates a location outside the security area. While the above embodiment and modification have been described with respect to two modes, normal mode and security mode, this is not a limitation. That is, the security mode may include, for example, multiple security modes (1 to 4 in the table below) according to security levels, as shown in Table 1 below, but is not limited to the combinations in Table 1. [Table 1]

[0144] 11A to 11D are rear views showing modified layouts of the rear camera unit. In the example of FIG. 11A, the rear camera 33 is disposed between the aimer light illumination unit 36 ​​and the distance measurement sensor 37 in the short-side direction Xs, and between the wide-angle illumination unit 34 and the narrow-angle illumination unit 35 in the long-side direction Xl. The distance K35 between the rear camera 33 and the narrow-angle illumination unit 35 is equal to or less than the distance K34 between the rear camera 33 and the wide-angle illumination unit 34.

[0145] 11B, the rear camera 33 is disposed between the aimer light illumination unit 36 ​​and the distance measurement sensor 37 in the short-side direction Xs. The wide-angle illumination unit 34 and the narrow-angle illumination unit 35 are disposed below the rear camera 33. In the short-side direction Xs, the narrow-angle illumination unit 35 is disposed on one side of the rear camera 33, and the wide-angle illumination unit 34 is disposed on the other side of the rear camera 33. The distance K35 between the rear camera 33 and the narrow-angle illumination unit 35 is equal to or less than the distance K34 between the rear camera 33 and the wide-angle illumination unit 34.

[0146] 11C , the wide-angle illuminator 34, the narrow-angle illuminator 35, the rear camera 33, the aimer light illuminator 36, and the distance measurement sensor 37 are arranged in this order in the short-side direction Xs. That is, the narrow-angle illuminator 35 is disposed between the rear camera 33 and the wide-angle illuminator 34, and the aimer light illuminator 36 is disposed between the rear camera 33 and the distance measurement sensor 37. Furthermore, the distance K35 between the rear camera 33 and the narrow-angle illuminator 35 is equal to or less than the distance K34 between the rear camera 33 and the wide-angle illuminator 34, and the distance K36 between the rear camera 33 and the aimer light illuminator 36 is equal to or less than the distance K37 between the rear camera 33 and the distance measurement sensor 37.

[0147] 11D , the wide-angle illuminator 34, rear camera 331, aimer-light illuminator 36, rear camera 33, narrow-angle illuminator 35, and distance measurement sensor 37 are arranged in this order in the short-side direction Xs. That is, the aimer-light illuminator 36 is disposed between the rear camera 33 and the wide-angle illuminator 34, and the rear camera 331 is disposed between the aimer-light illuminator 36 and the wide-angle illuminator 34. The narrow-angle illuminator 35 is disposed between the rear camera 33 and the distance measurement sensor 37. The distance K35 between the rear camera 33 and the narrow-angle illuminator 35 is equal to or less than the distance K34 between the rear camera 33 and the wide-angle illuminator 34 and is also equal to or less than the distance K37 between the rear camera 33 and the distance measurement sensor 37. The distance K36 between the rear camera 33 and the aimer-light illuminator 36 is equal to or less than the distance K34 between the rear camera 33 and the wide-angle illuminator 34.

[0148] Various differences in configuration are conceivable between rear camera 33 and rear camera 331. For example, rear camera 331 may be configured so that the imaging distance of rear camera 331 is greater than the imaging distance of rear camera 33. Alternatively, rear camera 331 may be configured so that the angle of the field of view at which rear camera 331 captures images is wider than the angle of the field of view at which rear camera 33 captures images.

[0149] Furthermore, the manner in which the image view window 93 is displayed together with the user application screen 91 can be changed as appropriate. FIGS. 12A and 12B are diagrams showing a first modified example of the display manner of the user application screen and the image view window. FIG. 12A shows the display on the display 20 before the image view window 93 is displayed, and FIG. 12B shows the display on the display 20 after the image view window 93 is displayed. In this first modified example, the service unit 88 acquires, from the user application 87, position information indicating the position on the user application screen 91 of the input item A3 that has been focused by the user's selection. The service unit 88 then displays the image view window 93 on the display 20 in a range outside the position of the focused input item A3 indicated by the position information.

[0150] That is, the service unit 88 receives, from the user application 87, position information on the user application screen 91 of the input item A3 that is focused as an input target. Then, based on the position information, the service unit 88 displays the image view window 93 at a position on the user application screen that does not overlap the focused input item A3 (FIG. 12B). In this configuration, the image view window 93 is displayed so as not to overlap the focused input item A3 on the user application screen 91, allowing the user to view the input item A3. In this way, it is possible to display the camera field of view V33 of the rear camera 33 while ensuring user operability of the user application screen 91.

[0151] Alternatively, the image view window 93 can be displayed in a display area specified by the user. Figures 13A and 13B show a second modified example of the display mode of the user application screen and the image view window. Figure 13A shows the display on the display 20 before the image view window 93 is displayed, and Figure 13B shows the display on the display 20 after the image view window 93 is displayed.

[0152] In this second modified example, the display area of ​​the image view window 93 can be specified on the display 20 by a user's operation on the display 20. This display area specification may be performed by setting the display area for the service unit 88 by operating the display 20, or by performing a predetermined operation (double tap or long touch) on a position on the display 20 that is to be specified as the display area. When the service unit 88 receives this specification of the display area, it displays the image view window 93 in that display area on the display 20, and also displays the user application screen 91 in an area outside the display area (FIG. 13B). FIG. 13B shows a case where the upper portion of the display 20 is specified as the display area.

[0153] That is, the service unit 88 receives a specification of the display area of ​​the image view window 93 on the display 20, and displays the user application screen 91 outside the display area of ​​the image view window 93 on the display 20. In this configuration, the position of the image view window 93 is determined in advance, and when the image view window 93 is displayed, the user application screen 91 is displayed outside of the image view window 93, allowing the user to view the focused input item A3. In this way, it is possible to display the camera field of view V33 of the rear camera 33 while ensuring user operability of the user application screen 91.

[0154] Furthermore, the control of the display period of the image view window 93 may be changed as appropriate. Fig. 14A is a flowchart showing a first modified example of the display control of the image view window. When the service unit 88 confirms that the trigger button 23 has been pressed ("YES" in step S1001), it displays the image view window 93 on the display 20 in the same manner as described above (step S1002) and causes the decoder 86 to start the decoding process (step S1003). Then, when the decoder 86 completes the decoding process (step S1004), the service unit 88 monitors whether the trigger button 23 has been pressed again (step S1005). When it confirms that the trigger button 23 has been pressed again ("YES" in step S1005), the service unit 88 ends the display of the image view window 93 (step S1006).

[0155] That is, when the trigger button 23 is operated, the service unit 88 causes the decoder 86 (image processing unit) to execute the decoding process (image processing), and when the trigger button 23 is operated again after the decoding process is completed, the service unit 88 causes the image view window 93 to close on the display 20. In this configuration, the image view window 93 displayed by operating the trigger button 23 remains displayed until the trigger button 23 is operated again. That is, the image view window 93 does not close immediately upon completion of the decoding process before the user has time to check the code C that is the target of the decoding process, and the user can reliably check the code C that is the target of the decoding process through the image view window 93.

[0156] 14B is a flowchart showing a second modified example of the display control of the image view window. In this second modified example, when an input item is focused by a user's designation operation on the display 20, the user application 87 notifies the service unit 88 of the focus of the input item. Then, when the service unit 88 receives the notification of the focus of the input item ("YES" in step S1101), it displays the image view window 93 on the display 20 (step S1102).

[0157] That is, the user application 87 notifies the service unit 88 that an input item has been specified via the display 20, and the service unit 88, upon receiving the notification from the user application 87, displays the image view window 93 on the display 20. In this configuration, the image view window 93 can be quickly displayed at the timing when the user wants to perform an operation related to the decoding process (that is, at the timing when the input item is specified).

[0158] Furthermore, the registration management of the read data Dc by the user application 87 may be configured as shown in Fig. 15. Fig. 15 is a flowchart showing an example of the registration management of read data by a user application. In step S1201, the user application 87 checks the difference between the number of read data Dc registered in the data management table 95 (number of registrations) and a predetermined specified number. Then, in step S1202, the user application 87 determines whether there is a shortage of the number of registrations relative to the specified number (in other words, it determines whether the number obtained by subtracting the number of registrations from the specified number is 1 or more).

[0159] If there is a shortage (YES in step S1202), the user application 87 transmits the shortage to the service unit 88 (step S1203), and the service unit 88 displays the shortage received from the user application 87 on the display 20 (for example, the image view window 93). Furthermore, a merge format for subsequent transmission of multiple scanned data Dc to the user application 87 is transmitted from the user application 87 to the service unit 88 (step S1204). Alternatively, the format may be defined in a setting file of the terminal prepared by the user. For example, this merge format indicates that multiple scanned data Dc are to be merged by separating them with commas. Therefore, when the service unit 88 confirms a registration instruction operation in the multiple code reading mode of FIG. 5A (step S105), it merges the multiple scanned data Dc that are the target of the registration instruction operation in accordance with the merge format and transmits the merged data to the user application 87. If there is no shortage (NO in step S1202), the user application 87 transmits the specified number of read data Dc to the external host computer (step S1205).

[0160] That is, the user application 87 specifies to the service unit a combination format for combining data to be transmitted in subsequent registration operations, based on the registration status of the user application 87 and the results of the read data Dc transmitted from the service unit 88 (steps S1201, S1204). Then, the service unit 88 combines the read data Dc, which is the result of the decoding process of the multiple codes C included in the image view window 93, based on the specified combination format, and transmits the combined data to the user application 87. In this configuration, the read data Dc, which is the result of the decoding process of the multiple codes C, can be combined in an appropriate combination format and transmitted to the user application 87 all at once.

[0161] Furthermore, when the cumulative total number of registered read data Dc (results of decoding processing) transmitted from the service unit 88 by the registration operation reaches a specified number ("NO" in step S1202), the user application 87 transmits the specified number of read data Dc to the external host (step S1205). In this configuration, the frequency of transmission from the user application 87 to the external host can be reduced, thereby reducing the load on them.

[0162] Furthermore, the user application 87 transmits the number of scan data Dc that is missing up to the specified number based on the number of registered scan data Dc and the specified number to the service unit 88 (step S1203), and the service unit 88 displays the missing number on the display 20. This configuration allows the user to easily check the missing number on the display 20. Furthermore, the service unit 88 can also recognize the progress of registration of the image processing results based on a response from the user application 87, not just when the information is sent from the user application 87. If the service unit 88 has previously acquired information on the specified number, it can also recognize the missing number. By displaying the progress of registration on the display 20, the service unit 88 can easily grasp the current number of scan data and the missing number. The service unit 88 may also acquire a determination condition regarding the timing of transmitting the image processing results registered in the user application 87 to the external host. The judgment conditions include, for example, but are not limited to, the number of read data items registered in the user application (if 10 items are registered), the code type (if a specific code type is read), and a user operation (if a double tap is performed). As with format information, the judgment conditions may not only be transmitted from the user application 87 to the service unit 88, but may also be defined in a setting file of the terminal prepared by the user. When the judgment conditions are met, the service unit 88 transmits to the user application 87 an instruction to transmit the registered image processing results from the user application 87 to an external host. This eliminates the need for the user application to implement logic for determining the timing of transmission to the external host.

[0163] Furthermore, the specific manner in which the administrator issues a security mode setting instruction to the security setting unit 74 can be modified as appropriate. That is, in the above embodiment, the security setting unit 74 displays an administrator setting screen on the display 20 and accepts the administrator's security mode setting instruction based on the administrator's operation on the administrator setting screen. In contrast, in the modified example described here, the security setting unit 74 is provided with a function for communicating with an external management server used by the administrator. The security setting unit 74 then receives a mode selection signal from the management server.

[0164] That is, the security setting unit 74 (communication unit) has a communication function for receiving a mode selection signal. The security setting unit 74 selects either the security mode or the normal mode based on the mode selection signal received by the communication function. Here, the mode selection signal indicates whether the security mode or the normal mode is to be selected based on a setting made by an administrator. This configuration can prevent users other than the administrator or finders from switching between the security mode and the normal mode.

[0165] In the above example, when reading a code, the narrow-angle illumination unit 35 irradiates the camera field of view V33 with narrow-angle illumination light L35, and the rear camera 33 captures an image of the code C within the camera field of view V33 to obtain a code image Ic. However, the wide-angle illumination unit 34 may also be used when reading a code. For example, it is possible that the code image Ic captured using the narrow-angle illumination light L35 may be overexposed, resulting in failure to read the read data Dc during decoding of the code image Ic. In such a case, the illumination unit used may be switched from the narrow-angle illumination unit 35 to the wide-angle illumination unit 34, and the code reading may be performed again. Specifically, the wide-angle illumination unit 34 irradiates the camera field of view V33 with wide-angle illumination light L34, and the rear camera 33 captures an image of the code C within the camera field of view V33 to obtain a code image Ic. The decoder 86 then decodes the code image Ic to obtain the read data Dc.

[0166] That is, the service unit 88 (imaging control unit) causes the narrow-angle illumination unit 35 to irradiate narrow-angle illumination light L35 while causing the private API control unit 56 (focus adjustment unit) to adjust the focus position of the rear camera 33, and if the decoding process on the code image Ic generated by having the rear camera 33 capture an image fails, the service unit 88 (imaging control unit) causes the wide-angle illumination unit 34 to irradiate wide-angle illumination light L34 while causing the private API control unit 56 (focus adjustment unit) to adjust the focus position of the rear camera 33, and then executes the decoding process on the code image Ic generated by having the rear camera 33 capture an image. With this configuration, if the decoding process using the narrow-angle illumination light L35 fails due to overexposure or the like, there is a possibility that the decoding process using the wide-angle illumination light L34 may be successful.

[0167] 2A, in order to prohibit photography in security mode, a public camera control API 51 and a private camera control API 52 are prepared and used selectively. However, it is also possible to provide only the private camera control API 52 and allow access to the camera module 3 only from a specific application such as the code reading processing unit 72 in security mode.

[0168] Furthermore, specific examples of image processing are not limited to the above-mentioned decoding processing, but also include other image processing such as OCR. [Industrial Applicability]

[0169] The present invention is applicable to all techniques for reading codes attached to works. [Explanation of symbols]

[0170] 1. Mobile information terminal 11...Housing 12...Front 13...Back 14...Right side 141...recess 15...Left side 16…Top surface 17…Bottom surface 20...Display 21...Volume button 22...Volume button 23...Trigger button 231...Foundation part 232...Protrusion 3...Camera module 31...Front camera 32...Rear camera unit 33...Rear camera 331...Rear camera 34...Wide-angle lighting section 35...Narrow-angle lighting section 36...Aimer light illumination unit 37...Range measurement sensor 38…Near field communication department 39...Start control unit 4...Attachment 41...Attachment cover 413...Window 414...Window 415...Window 416...Window 417...Window 423...Polarizing plate 425...Polarizing plate 43...Light diffuser 44...Prism mirror 45...Color filter 5...Control unit 51...Camera control public API 52...Camera control private API 53...API execution switching control unit 54...Camera control unit 55...Public API control unit 56...Controller for non-public API 61...Volatile memory unit 62...Non-volatile memory unit 71...Photography processing unit 72...Code reading processing unit 73…Soc 731...ISP 732…DSP 733...GPU 734...CPU 74...Security settings section 75...Prohibited Section 76...Screenshot Prohibition Section 77...Screenshot control section 78...Position information calculation section 81...Communication control unit 82…Internet Communications Department 83...Telephone and Communications Department 84…Call Recording Section 86...Decoder 87...User application 88...Service Department 91...User application screen 911...Input field 912...Data display column 93...Image View Window 94...Virtual Aim 95...Data management table 951...Data registration conditions 952...Reading result 96…Depth of field information 961…Position coordinates 962…Depth of field 963...Focus position C...chord Dc...Read data E… Gaze Ic...Code image Il... Image View Image Ip...Photo image J...Scope K34…distance K35…distance K36…distance K37…distance L34...Wide-angle illumination L35...narrow angle illumination L36...Aima light L37…Distance measurement light M11…center M33…center Ma…Mark Mb…Mark Mj…center R…Spacing V33…Camera field of view W…Work Xl...Longitudinal direction Xs…Short side direction Xt: Thickness direction Δf…interval Δn…interval

Claims

1. a camera that captures an image of a workpiece and generates a captured image; an image processing unit that performs image processing based on the captured image; a service unit that has at least one input item, transmits the result of the image processing to a user application that reflects the result of the image processing in the input item, and receives a response from the user application; a display unit that displays an image view window showing the captured image; Equipped with the user application determines whether or not the result of the image processing needs to be registered in the user application based on the result of the image processing received from the service unit, and transmits the determination result of whether or not the result of the image processing needs to be registered to the service unit; The service unit displays on the display unit the image view window showing the information indicating the determination result received from the user application superimposed on the captured image.

2. 2. The portable information terminal according to claim 1, wherein the user application has at least one condition regarding the result of the image processing that needs to be registered in the user application, and determines whether or not the registration is necessary based on the condition.

3. The mobile information terminal according to claim 2 , wherein the conditions include a condition relating to any one of a type of the target of the image processing, a character string included in the target of the image processing, a numerical value included in the target of the image processing, and a position of the target of the image processing.

4. The mobile information terminal described in claim 1, wherein the service unit causes the display unit to display an image view window in which targets of image processing contained in the captured image being displayed in the image view window that need to be registered in the user application are identified based on the judgment result received from the user application.

5. The mobile information terminal according to claim 4, wherein the service unit causes the display unit to display the image view window, which distinguishes between objects for image processing that require registration, objects that are already registered and do not require registration, and objects that are not registered but do not require registration.

6. A mobile information terminal as described in claim 4 or 5, wherein when the service unit accepts a registration operation by a user while the image view window in which the object requiring registration is identified is displayed on the display unit, the service unit registers the results of image processing related to the object requiring registration in the user application.

7. The service department obtaining a format for combining data transmitted in the registration operation; 7. The portable information terminal according to claim 6, wherein the results of the image processing of the plurality of objects included in the image view window are combined based on the format and transmitted to the user application.

8. The service department acquire a determination condition regarding the timing for transmitting the image processing result registered in the user application to an external host; 7. The portable information terminal according to claim 6, wherein when the determination condition is satisfied, an instruction for transmitting the registered image processing result from the user application to an external host is transmitted to the user application.

9. 7. The portable information terminal according to claim 6, wherein the service unit recognizes a progress status of registration of the result of the image processing based on the response from the user application, and causes the display unit to display the progress status of registration.

10. A step in which a camera captures an image of a workpiece and generates a captured image; performing image processing based on the captured image; a step in which a service unit transmits the result of the image processing to a user application having at least one input item and reflecting the result of the image processing in the input item, and the service unit receives a response from the user application; displaying an image view window showing the captured image on a display unit; Equipped with the user application determines whether or not the result of the image processing needs to be registered in the user application based on the result of the image processing received from the service unit, and transmits the determination result of whether or not the result of the image processing needs to be registered to the service unit; The image processing result registration method includes causing the service unit to display, on the display unit, the image view window that shows information indicating the determination result received from the user application superimposed on the captured image.

Images