Control device

Abstract

This control device comprises: a detection unit that displays a reading target in accordance with image data including the reading target as a subject, and detects, from the image data, a predetermined mark allowing specification of a control target to which an imaging device is to be directed, the imaging device acquiring the image data; a specifying unit that specifies a control target from the image data in accordance with the mark detected by the detection unit; and a control unit that controls the imaging device such that the imaging direction of the imaging device faces the control target specified by the specifying unit.

Description

control device

[0001] The present invention relates to a control device, a control method, and a recording medium.

[0002] 2. Description of the Related Art Techniques relating to information codes such as QR Codes (registered trademark) are known.

[0003] For example, Patent Document 1 discloses an information display that displays an information code that can be read by a reading device. According to Patent Document 1, the information code includes specific information that identifies the installation location where the information display is to be displayed, and the specific information is displayed in a manner that is visibly recognizable other than linguistic information. For example, Patent Document 1 discloses that, in addition to the color of a QR code (registered trademark), a pattern or design corresponding to the specific information may be displayed in a space other than the reading area of ​​the information code as a display form that visibly displays the specific information.

[0004] Japanese Patent Application Laid-Open No. 2020-177543

[0005] When reading an information code while in a vehicle, for example, it is sometimes necessary to read the information code from a long distance using an imaging device such as a PTZ (pan-tilt-zoom) camera. In such cases, it is desirable to detect an information processing device, such as a smartphone, that displays the information code from the acquired image data, control the orientation of the imaging device relative to the detected information processing device, and then read the information code using the image data acquired after control. In this case, simply performing the control based on the results of object detection such as YOLO (You Only Look Once) can result in erroneous detection of an information processing device other than the target, potentially causing the imaging device to be controlled in the wrong direction. As a result, controlling the imaging device to read an object such as an information code can be time-consuming.

[0006] These problems cannot be solved even by using the technology described in Patent Document 1. Therefore, one of the objects of the present invention is to provide a control device, a control method, and a recording medium that can solve the above-mentioned problems.

[0007] In order to achieve this object, a control device that is one form of the present disclosure has a configuration including: a detection unit that detects, in accordance with image data including a read target as a subject, a predetermined mark from the image data that can identify a control target that is an object to which an imaging device that displays the read target and acquires the image data is directed; an identification unit that identifies the control target from the image data in accordance with the mark detected by the detection unit; and a control unit that controls the imaging device so that the imaging direction of the imaging device faces the direction of the control target identified by the identification unit.

[0008] Furthermore, a control method according to another aspect of the present disclosure is configured such that an information processing device, in accordance with image data including a reading target as a subject, detects from the image data a predetermined landmark capable of identifying a control target to which an imaging device that displays the reading target and acquires the image data is directed, identifies the control target from the image data in accordance with the detected landmark, and controls the imaging device so that the imaging direction of the imaging device faces the identified control target.

[0009] Furthermore, a recording medium that is another form of the present disclosure is a computer-readable recording medium having recorded thereon a program for causing an information processing device to perform the following processes: detect, in accordance with image data including a reading target as a subject, a predetermined landmark from the image data that can identify a control target to which an imaging device that displays the reading target and acquires the image data is directed; identify the control target from the image data in accordance with the detected landmark; and control the imaging device so that the imaging direction of the imaging device faces the direction of the identified control target.

[0010] According to the above-described configurations, when controlling the imaging device to read an information code, the time required for reading can be reduced.

[0011] 1 is a diagram illustrating an example configuration of an entire control system. FIG. 2 is a block diagram illustrating an example configuration of a control device. FIG. 3 is a diagram illustrating an example processing of a control device. FIG. 4 is a diagram illustrating an example processing of a candidate specification unit. FIG. 5 is a diagram illustrating an example processing of a landmark detection unit. FIG. 6 is a diagram illustrating an example processing of a control target specification unit. FIG. 7 is a diagram illustrating an example processing of a focus adjustment unit. FIG. 8 is a flowchart illustrating an example operation of a control device. FIG. 9 is a diagram illustrating an example hardware configuration of a control device in a second embodiment of the present disclosure. FIG. 10 is a block diagram illustrating an example configuration of a control device. FIG. 11 is a flowchart illustrating an example operation of a control device.

[0012] [First Embodiment] A first embodiment of the present invention will be described with reference to Figs. 1 to 9. Fig. 1 is a diagram illustrating an example of the overall configuration of a control system 100. Fig. 2 is a block diagram illustrating an example of the configuration of a control device 300. Fig. 3 is a diagram illustrating an example of the processing performed by the control device 300. Fig. 4 is a diagram illustrating an example of the processing performed by a candidate identification unit 353. Fig. 5 is a diagram illustrating an example of the processing performed by a landmark detection unit 354. Fig. 6 is a diagram illustrating an example of the processing performed by a control target identification unit 355. Figs. 7 and 8 are diagrams illustrating an example of the processing performed by a focus adjustment unit 357. Fig. 9 is a flowchart illustrating an example of the operation of the control device 300. Note that in this disclosure, the drawings may be associated with one or more of the embodiments.

[0013] In a first embodiment of the present disclosure, a control system 100 is described that controls an imaging device 200, such as a PTZ (pan-tilt-zoom) camera, to acquire image data and read an information code from a long distance. For example, when a system user holds up an information processing terminal displaying an information code, the control system 100 uses the imaging device 200 to acquire image data of the user's appearance. In other words, the control system 100 acquires image data including the information code to be read as the object. Furthermore, upon acquiring image data from the imaging device 200, the control system 100 uses the acquired image data to detect a landmark for identifying the information processing terminal displaying the information code to be read. In other words, the control system 100 detects a landmark for identifying the information processing terminal to be controlled, such as by pointing the imaging direction of the imaging device 200. The control system 100 then identifies the control target from the image data based on the detected landmark. For example, the control system 100 identifies, among rectangular objects included in the image data, a rectangular object on which a detected landmark is displayed or a rectangular object that is the shortest distance from the detected landmark as a control target. The control system 100 then controls the imaging device 200 so that the identified control target is located at the center of the image data, and then controls the imaging device 200 so that the size of the information code to be read is increased to a size necessary for reading. In other words, the control system 100 controls the pan / tilt function of the imaging device 200 so that the identified control target is located at the center of the image data acquired by the imaging device 200, and then controls the zoom function of the imaging device 200 until the size of the identified control target is large enough.

[0014] Furthermore, before identifying a control target based on a landmark, the control system 100 can identify candidates based on the skeleton of a person detected from image data. For example, the control system 100 detects the skeleton of a person from the acquired image data. Then, the control system 100 identifies candidate control targets from rectangular objects, etc., included in the image data based on the detected skeleton. For example, the control system 100 identifies, as candidates, rectangular objects included in the image data whose distance from the detected skeleton is equal to or less than a predetermined value, or rectangular objects whose distance from a skeleton that can be determined to be in a predetermined posture, such as a posture in which an information processing terminal is being held up, etc. In this case, the control system 100 may identify a control target from the candidate control targets identified based on the detected landmark.

[0015] Furthermore, after controlling the zoom function of the imaging device 200, the control system 100 can adjust the focus using image data acquired after zooming. For example, the control system 100 acquires a profile of a predetermined location in an information code, such as a finder pattern for position detection in a QR code (registered trademark). In other words, the control system 100 acquires information indicating the outline of the information code, such as changes in pixel values ​​at a predetermined location in the information code. The control system 100 also calculates the derivative of the acquired information to determine whether the width ε of the identifiable convexity is equal to or less than a predetermined value. The control system 100 then controls the focus of the imaging device 200 so that the width ε of the convexity is equal to or less than the predetermined value. For example, by performing the above-described process, the control system 100 controls the zoom function and then focuses on the information code to be read. The control system 100 can then read the information code using the image data acquired after focusing.

[0016] In the present disclosure, the term "mark" refers to any mark capable of identifying the control target, which is the target to which the imaging device 200 is aimed, and may include any object, color, pattern, etc. For example, as described below, the mark may be a pattern or color displayed in a blank area, separate from the display area of ​​the information code, on an information processing terminal or the like that displays the information code. For example, the mark may be any pattern or color that can be identified from a long distance, such as a black circle or red. Furthermore, the mark may be displayed together with the information code on the information processing terminal or the like that displays the information code, or may be a predetermined object such as a vehicle side mirror or steering wheel. For example, if it is predetermined that the person sitting in the driver's seat will display the information code, the side mirror or steering wheel may be used as the mark. In addition to the above examples, the mark may be determined as appropriate depending on prior agreement regarding the person who will display the information code.

[0017] Furthermore, the information code to be read as described in this disclosure is, for example, a two-dimensional code such as a QR code (registered trademark). The read object may be a one-dimensional code such as a barcode, or any other information. Furthermore, this disclosure describes a case in which the information code to be read is displayed on an information processing terminal such as a smartphone or tablet terminal. That is, the case in which the control object is an information processing terminal such as a smartphone or tablet terminal is described. However, the information code may be displayed by a method other than displaying it on an information processing terminal, such as being printed on paper. Therefore, the control object is not necessarily limited to an information processing terminal, and may also be paper on which an information code is displayed.

[0018] Fig. 1 shows an example of the overall configuration of a control system 100. Referring to Fig. 1, the control system 100 includes an imaging device 200 and a control device 300. As shown in Fig. 1, the imaging device 200 and the control device 300 are connected to each other so as to be able to communicate with each other via wire or wirelessly.

[0019] 1 illustrates a case where an information code displayed on an information processing terminal held up by a person inside a vehicle is read using the image capture device 200 and the control device 300. However, the present invention is not limited to cases where a person is inside a vehicle, and can be applied to various situations, such as any situation where the image capture device 200 is controlled to read an information code. For example, the control system 100 may be configured to read an information code displayed on an information processing terminal held up by a person walking, etc.

[0020] The imaging device 200 is a camera such as a PTZ camera whose imaging direction can be controlled. For example, the imaging device 200 has a pan / tilt function for swinging the camera in the horizontal and vertical directions, and a zoom function for enlarging and reducing the imaging screen, in response to control from the control device 300.

[0021] For example, the imaging device 200 acquires time-series image data at predetermined intervals. The imaging device 200 may continue to acquire image data at predetermined intervals while changing the imaging direction or adjusting the focus under control of the control device 300 (described later). The imaging device 200 can also transmit the acquired image data to the control device 300.

[0022] The control device 300 is an information processing device that controls the imaging device 200, such as controlling the imaging direction and zoom function of the imaging device 200 and adjusting the focus, and also reads information codes in accordance with image data acquired by the imaging device 200. Referring to Fig. 2, the control device 300 has, as main components, for example, an operation input unit 310, a screen display unit 320, a communication interface unit 330, a storage unit 340, and an arithmetic processing unit 350.

[0023] 2 illustrates an example in which the functions of the control device 300 are realized using one information processing device. However, at least some of the functions of the control device 300 may be realized using multiple information processing devices, for example, on the cloud. Furthermore, the control device 300 may not include some of the components illustrated above, such as not having the operation input unit 310 or the screen display unit 320, or may have a configuration other than those illustrated above.

[0024] The operation input unit 310 is made up of operation input devices such as a keyboard, a mouse, etc. The operation input unit 310 detects operations of the operator operating the control device 300 and outputs the operations to the arithmetic processing unit 350.

[0025] The screen display unit 320 is composed of a screen display device such as a liquid crystal display, an organic electroluminescence (EL) display, etc. The screen display unit 320 can display various information stored in the storage unit 340 on the screen in response to instructions from the arithmetic processing unit 350.

[0026] The communication interface unit 330 is composed of a data communication circuit, etc. The communication interface unit 330 performs data communication with an external device such as the image capture device 200 connected via a communication line.

[0027] The storage unit 340 is a storage device such as a hard disk or memory. The storage unit 340 stores processing information and a program 342 required for various processes in the arithmetic processing unit 350. The program 342 is read into the arithmetic processing unit 350 and executed to realize various processing units. The program 342 is read in advance from an external device or recording medium via a data input / output function such as the communication interface unit 330, and is stored in the storage unit 340. Main information stored in the storage unit 340 includes, for example, image information 341.

[0028] The image information 341 includes image data acquired by the imaging device 200. For example, the image information 341 includes time-series image data acquired by the imaging device 200. The image information 341 is updated in response to the image data acquisition unit 351 acquiring image data from the imaging device 200, etc.

[0029] The arithmetic processing unit 350 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 350 reads and executes a program 342 from the storage unit 340, thereby causing the above hardware and the program 342 to cooperate to realize various processing units. Major processing units realized by the arithmetic processing unit 350 include, for example, an image data acquisition unit 351, a skeleton detection unit 352, a candidate identification unit 353, a landmark detection unit 354, a control target identification unit 355, an imaging device control unit 356, a focus adjustment unit 357, and a reading unit 358.

[0030] In addition, instead of the above-mentioned CPU, the arithmetic processing unit 350 may have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these.

[0031] The image data acquisition unit 351 acquires image data acquired by the imaging device 200 from the imaging device 200. The image data acquisition unit 351 also stores the acquired image data as image information 341 in the storage unit 340.

[0032] 3, the skeleton detection unit 352 detects the skeleton of a person included in the image data in accordance with the image data. For example, the skeleton detection unit 352 can detect the skeleton of a person by inputting the image data into a skeleton detection model that has been trained in advance.

[0033] For example, the skeleton detection unit 352 detects each part of the person corresponding to at least the upper body, such as the shoulders, elbows, and wrists, in response to inputting image data into a skeleton detection model, and acquires information indicating the coordinates of each part in the image data. The skeleton detection unit 352 may also be configured to detect parts other than the upper body.

[0034] As will be described later, the skeleton detection unit 352 may be configured to detect the skeleton and also detect a posture that can be determined based on the detected skeleton. For example, the skeleton detection unit 352 may detect the skeleton and posture based on the image data by inputting the image data into a model that has been trained in advance to detect the skeleton and posture.

[0035] The candidate identification unit 353 identifies, from the image data, a candidate for a control target to which the imaging direction of the imaging device 200 is directed, according to the detection result by the skeleton detection unit 352. For example, the candidate identification unit 353 detects a rectangular object according to the image data, and identifies, from the detected rectangular objects, a candidate for a control target according to the detection result by the skeleton detection unit 352. As an example, the candidate identification unit 353 can identify a candidate for a control target according to the distance between the rectangular object detected according to the image data and the skeleton of a person detected by the skeleton detection unit 352.

[0036] For example, the candidate identification unit 353 detects rectangular objects in image data using an object detection model such as YOLO (You Only Look Once). Furthermore, the candidate identification unit 353 identifies rectangular objects that satisfy predetermined conditions as candidates for control targets. For example, the candidate identification unit 353 identifies rectangular objects whose distance from a predetermined part, such as an elbow or wrist, is less than a predetermined value among the coordinates of each part corresponding to the skeleton detected by the skeleton detection unit 352 as candidates for control targets. Furthermore, the candidate identification unit 353 may perform the above identification based on the distance from the skeleton that can be determined to be a predetermined posture, such as a posture holding up an information processing terminal. For example, the candidate identification unit 353 determines whether the person's posture is a predetermined posture, such as a posture holding up an information processing terminal, by checking whether the coordinates of each part detected by the skeleton detection unit 352 satisfy predetermined conditions. Thereafter, the candidate identification unit 353 identifies, as a candidate for control action, a rectangular object whose distance to a specific part of the skeleton, such as an elbow or wrist, that can be determined to be a specific posture of the person is equal to or less than a specific value.

[0037] For example, in the case shown in Fig. 3, the candidate identification unit 353 detects rectangular objects r1, r2, and r3 from the image data using an object detection model. In the example shown in Fig. 3, the distance between the rectangular objects r1 and r2 and a predetermined part, such as an elbow or a wrist, detected by the skeleton detection unit 352 is less than a predetermined value. On the other hand, the distance between the rectangular object r3 and a predetermined part, such as an elbow or a wrist, detected by the skeleton detection unit 352 exceeds a predetermined value. Therefore, as shown in Fig. 4, the candidate identification unit 353 identifies the rectangular objects r1 and r2 from among the rectangular objects r1, r2, and r3 in the image data as candidates for control targets, in accordance with the detection result by the skeleton detection unit 352.

[0038] The candidate identification unit 353 may determine the posture by inputting the coordinates of each body part detected by the skeleton detection unit 352 into a pre-trained model. Furthermore, the posture may be determined by the skeleton detection unit 352 instead of by the candidate identification unit 353, as described above. In this case, the candidate identification unit 353 may identify candidates for the control target using the posture detected by the skeleton detection unit 352.

[0039] Furthermore, the candidate identification unit 353 may identify candidates based on the detection results of a person's face, instead of the skeleton or posture. For example, the candidate identification unit 353 may be configured to detect a person's face based on image data and identify candidates by checking whether the detected person's face is facing forward. For example, the candidate identification unit 353 can identify candidates based on the distance between a face facing forward and a rectangular object.

[0040] The mark detection unit 354 detects a predetermined mark for identifying the control target according to the image data.

[0041] For example, the mark detection unit 354 detects a mark that is a predetermined color or pattern as a mark. For example, as shown in FIG. 5 , when an information code is displayed on an information processing terminal or the like, a predetermined pattern or color is displayed in a blank area separate from the display area of ​​the information code. The mark detection unit 354 can detect the mark displayed as described above by detecting the color or pattern present in the rectangular object that is a candidate for control identified by the candidate identification unit 353 from the image data. The mark detection unit 354 may detect at least one of a predetermined color or pattern as a mark. Furthermore, the mark detection unit 354 may detect a predetermined object present outside the rectangular object, such as a vehicle side mirror or steering wheel, as a mark instead of or in addition to the color or pattern. The mark detection unit 354 may detect any predetermined object as a mark, in addition to the side mirror. The mark detection unit 354 may detect a mark using any means, such as using a predetermined model.

[0042] 3 to 5, the mark detection unit 354 can detect a mark m1 that is a predetermined color, pattern, or the like as a mark according to the image data. The mark detection unit 354 may also detect a mark m2 that is a predetermined object as a mark together with or instead of the mark m1.

[0043] The landmark detection unit 354 may be configured to detect landmarks other than those exemplified above. For example, the landmark detection unit 354 may be configured to detect a person's skeleton or the like as a landmark by using the processing results of the above-described skeleton detection unit 352 or by performing processing similar to that of the skeleton detection unit 352. The landmark detection unit 354 may also be configured to detect a person's face, such as a person looking straight ahead, as a landmark. For example, the landmark detection unit 354 may be configured to detect a person's face based on image data and detect a landmark by checking whether the detected person's face is facing forward.

[0044] The control target identification unit 355 identifies a control target from the image data using the landmark detected by the landmark detection unit 354. For example, the control target identification unit 355 can identify a control target from the control target candidates identified by the candidate identification unit 353 using the landmark detected by the landmark detection unit 354.

[0045] For example, the control target identification unit 355 identifies as the control target a candidate for which a landmark is detected inside the rectangle by the landmark detection unit 354, from among the candidates for control targets identified by the candidate identification unit 353. Furthermore, the control target identification unit 355 may identify as the control target a candidate for which the distance to the landmark detected by the landmark detection unit 354 is the shortest, from among the candidates for control targets identified by the candidate identification unit 353.

[0046] For example, in the example shown in FIG. 4 , rectangular objects r1 and r2 in the image data are identified by the candidate identification unit 353 as candidates for control targets. Also, as shown in FIG. 4 , a mark m1 is detected by the landmark detection unit 354 inside rectangular object r2, one of the candidates. Alternatively, the distance between rectangular object r2 and the mark m2 detected by the landmark detection unit 354 is shorter than that between rectangular object r1 and rectangular object r2. Therefore, as shown in FIG. 6 , the control target identification unit 355 identifies rectangular object r2, inside which a mark m1 is detected, as the control target, among the candidate control targets. Alternatively, the control target identification unit 355 identifies rectangular object r2, the object that is the shortest distance from the mark m2, as the control target, among the candidate control targets. Note that in cases where different control targets can be identified according to each of multiple landmarks, the control target identification unit 355 may identify the control target according to, for example, a predetermined priority of the landmarks.

[0047] The imaging device control unit 356 controls the imaging device 200 in accordance with the result specified by the control target specifying unit 355. For example, the imaging device control unit 356 controls the pan / tilt function of the imaging device 200 in accordance with the result specified by the control target specifying unit 355, and then controls the zoom function of the imaging device 200.

[0048] For example, the imaging device control unit 356 controls the pan / tilt function of the imaging device 200 in accordance with the result of identification by the control target identification unit 355 so that the imaging direction of the imaging device 200 faces the direction of the control target identified by the control target identification unit 355. In other words, the imaging device control unit 356 controls the pan / tilt function of the imaging device 200 so that the identified control target is located at the center of the image data acquired by the imaging device 200. Furthermore, after the above control, the imaging device control unit 356 controls the zoom function of the imaging device 200. For example, the imaging device control unit 356 controls the zoom function of the imaging device 200 until the size of the control target identified by the control target identification unit 355 becomes sufficiently large in the image data acquired by the imaging device 200. Note that the extent of zooming may be determined arbitrarily.

[0049] The focus adjustment unit 357 adjusts the focus using image data acquired by the imaging device 200 after zooming, after the imaging device control unit 356 has controlled the zoom function. For example, a simple zoomed state may result in an image being out of focus, and if an attempt is made to read an information code in such a state, the reading process may take an unnecessary amount of time. Therefore, the focus adjustment unit 357 adjusts the focus using image data acquired by the imaging device 200 after zooming, after the imaging device control unit 356 has controlled the zoom function.

[0050] FIG. 7 shows an example of a profile of the finder pattern when the image is out of focus, and FIG. 8 shows an example of a profile of the finder pattern when the image is in focus. As shown in FIG. 7 , when the image is out of focus, by acquiring changes in pixel values ​​corresponding to each position along the finder pattern as indicated by the arrows in FIG. 7 and calculating their derivatives, the width ε of the convexity exceeds a predetermined value, as shown in FIG. 7 . On the other hand, as shown in FIG. 8 , when the image is in focus, by acquiring changes in pixel values ​​corresponding to each position along the finder pattern and calculating their derivatives, the width ε of the convexity is approximately 0. Therefore, the focus adjustment unit 357 specifies the width ε of the convexity by acquiring changes in pixel values ​​at a predetermined position using image data acquired by the imaging device 200 after zooming and calculating their derivatives. Furthermore, the focus adjustment unit 357 performs focus adjustment by adjusting the focus value of the imaging device 200 so that the specified width ε of the convexity is equal to or less than a predetermined value, such as 0.

[0051] The reading unit 358 reads the information code to be read using the image data acquired by the imaging device 200. For example, the reading unit 358 can read the information code using the image data acquired by the imaging device 200 after the focus adjustment by the focus adjustment unit 357.

[0052] The above is an example of the configuration of the control device 300. Next, an example of the operation of the control device 300 will be described with reference to FIG. 9 . The process illustrated in FIG. 9 can be started, for example, when it is detected that a vehicle has entered a predetermined area. Here, the predetermined area is, for example, the area where immigration inspection is performed at an immigration gate. Intrusion into such an area may be detected using a sensor installed at the immigration gate. The sensor may be an imaging device such as a camera, a proximity sensor, or a weight sensor. The flow illustrated in FIG. 9 may be executed immediately after intrusion is detected, or may be executed after executing a predetermined process, such as a process of requesting the display of an information code, after intrusion is detected. The process illustrated in FIG. 9 may be started at any timing other than the above example.

[0053] 9 is a flowchart showing an example of the operation of the control device 300. Referring to FIG. 9, the skeleton detection unit 352 detects the skeleton of a person included in the image data based on the image data (step S101). For example, the skeleton detection unit 352 may detect the skeleton of a person by inputting the image data into a skeleton detection model that has been trained in advance.

[0054] The candidate specifying unit 353 specifies, from the image data, candidates for a control target to which the imaging direction of the imaging device 200 is directed, according to the detection result by the skeleton detection unit 352 (step S102). For example, the candidate specifying unit 353 detects a rectangular object according to the image data, and specifies, from the detected rectangular objects, candidates for a control target according to the detection result by the skeleton detection unit 352.

[0055] The mark detection unit 354 detects a predetermined mark for identifying the control target according to the image data (step S103). For example, the mark detection unit 354 can detect a color, a pattern, a predetermined object, or the like that exists in a predetermined location as the mark.

[0056] The control target specifying unit 355 uses the landmark detected by the landmark detection unit 354 to specify a control target from among the control target candidates specified by the candidate specifying unit 353 (step S104).

[0057] The imaging device control unit 356 controls the imaging device 200 in accordance with the result specified by the control target specifying unit 355 (step S105). For example, the imaging device control unit 356 controls the pan / tilt function of the imaging device 200 in accordance with the result specified by the control target specifying unit 355, and then controls the zoom function of the imaging device 200.

[0058] The focus adjustment unit 357 acquires a change in pixel value at a predetermined position using image data acquired by the imaging device 200 after zooming and calculates its derivative to identify the width ε of the convexity (step S106). If the width ε of the convexity exceeds a predetermined value (step S107, No), the focus adjustment unit 357 adjusts the focus value of the imaging device 200 (step S108). Then, the process returns to step S106. If the width ε of the convexity is equal to or smaller than a predetermined value (step S107, Yes), the reading unit 358 reads the information code using the image data acquired by the imaging device 200 after the focus adjustment by the focus adjustment unit 357 (step S109).

[0059] The above is an example of the operation of the control device 300.

[0060] As described above, the control device 300 includes a landmark detection unit 354, a control target identification unit 355, and an imaging device control unit 356. With this configuration, the imaging device control unit 356 can control the imaging device 200 so that the imaging direction faces the control target identified by the control target identification unit 355 in accordance with the landmark detected by the landmark detection unit 354. For example, if a rectangular object such as an information processing terminal is detected simply using YOLO or the like, it is impossible to distinguish between the information processing terminal displaying the information code to be read and other rectangular objects. As a result, there is a risk of controlling the imaging device 200 in the wrong direction. On the other hand, according to the present disclosure, since the control target is identified according to the landmark as described above, the risk of controlling the imaging device 200 in the wrong direction can be reduced. This reduces the time required for reading.

[0061] The control device 300 also has a skeleton detection unit 352 and a candidate identification unit 353. With this configuration, the control target identification unit 355 can identify a control target from among the candidates identified by the candidate identification unit 353 in accordance with the detection result by the skeleton detection unit 352. As a result, the control target can be more appropriately identified, and the time required for reading can be more appropriately reduced.

[0062] The control device 300 also includes a focus adjustment unit 357 and a reading unit 358. With this configuration, the reading unit 358 can read the information code using image data acquired after adjustment by the focus adjustment unit 357. Simply attempting to read the information code when the image is out of focus after zooming can result in unnecessary effort and time. By reading the information code after focus adjustment as in the present disclosure, the time required for reading can be more appropriately reduced.

[0063] In the present disclosure, the control device 300 identifies a control target from among control target candidates. However, the control device 300 may be configured to identify a control target based on a landmark detected from a rectangular object in image data, without identifying candidates. In such a configuration, the control device 300 may not have the functions of the skeleton detection unit 352 or the candidate identification unit 353. Furthermore, the control target identification unit 355 may have a function for performing object detection such as YOLO.

[0064] Furthermore, the control system 100 described in the present disclosure can be used to achieve faster immigration inspections, for example, when reading optional information such as passport information using an information code while an entrant is inside a vehicle. In addition to the above-described example, the control system 100 may be applied to various situations where it is necessary to control the image capture device 200 when reading an information code.

[0065] In addition, in the present disclosure, a case has been described in which the control system 100 includes the imaging device 200 and the control device 300. However, the imaging device 200 may have at least some of the functions of the control device 300. Furthermore, if the imaging device 200 has the functions of the control device 300 described in the present disclosure, the control system 100 may be composed of only the imaging device 200.

[0066] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 10 to Fig. 12. Fig. 10 is a diagram illustrating an example of the hardware configuration of a control device 400. Fig. 11 is a block diagram illustrating an example of the configuration of the control device 400. Fig. 12 is a flowchart illustrating an example of the operation of the control device 400.

[0067] In the second embodiment of the present disclosure, a control device 400 will be described, which is an information processing device that controls an imaging device for the purpose of reading a read target such as an information code. Fig. 10 shows an example of the hardware configuration of the control device 400. Referring to Fig. 10, the control device 400 has the following hardware configuration, for example: a CPU (Central Processing Unit) 401 (arithmetic unit); a ROM (Read Only Memory) 402 (storage device); a RAM (Random Access Memory) 403 (storage device); a group of programs 404 loaded into the RAM 403; a storage device 405 that stores the group of programs 404; a drive device 406 that reads and writes data from and to a storage medium 410 external to the information processing device; a communication interface 407 that connects to a communication network 411 external to the information processing device; an input / output interface 408 that inputs and outputs data; and a bus 409 that connects the various components.

[0068] 11 by the CPU 401 acquiring and executing the program group 404. The program group 404 is stored in advance in the storage device 405 or the ROM 402, for example, and is loaded into the RAM 403 or the like by the CPU 401 for execution as needed. The program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in advance in the recording medium 410, and the drive device 406 may read out the program and supply it to the CPU 401.

[0069] 10 shows an example of the hardware configuration of the control device 400. The hardware configuration of the control device 400 is not limited to the above-described case. For example, the control device 400 may be configured with only a part of the above-described configuration, such as excluding the drive device 406. Furthermore, the CPU 401 may be a GPU, etc., as exemplified in the first embodiment.

[0070] The detection unit 421 acquires image data that includes a read target such as an information code as a subject. Then, in accordance with the acquired image data, the detection unit 421 detects a predetermined mark from the image data that can identify a control target, which is an object to which an imaging device that displays the read target and acquires the image data is directed.

[0071] The identification unit 422 identifies a control target from the image data according to the mark detected by the detection unit 421. For example, the identification unit 422 identifies, from among the rectangular objects detected from the image data, a rectangular object in which a mark has been detected by the detection unit 421 as a control target.

[0072] The control unit 423 controls the imaging device that acquires image data so that the imaging direction of the imaging device faces the direction of the control target identified by the identification unit 422.

[0073] The above is an example of the configuration of the control device 400. Next, an example of the operation of the control device 400 will be described with reference to FIG.

[0074] Fig. 12 is a flowchart showing an example of the operation of the control device 400. Referring to Fig. 12, the detection unit 421 detects, in accordance with image data that includes the read target as a subject, a predetermined mark from the image data that can identify a control target, which is an object toward which an imaging device that displays the read target and acquires image data is directed.

[0075] The identification unit 422 identifies a control target from the image data according to the landmark detected by the detection unit 421 .

[0076] The control unit 423 controls the imaging device that acquires image data so that the imaging direction of the imaging device faces the direction of the control target identified by the identification unit 422.

[0077] The above is an example of the operation of the control device 400.

[0078] As described above, the control device 400 includes a detection unit 421, an identification unit 422, and a control unit 423. With this configuration, the control unit 423 can control the imaging device that acquires image data so that the imaging direction of the imaging device faces the direction of the control target identified by the identification unit 422 in accordance with the landmark detected by the detection unit 421. As a result, it is possible to reduce the risk of controlling the imaging device in the wrong direction of the target. This reduces the time required for reading.

[0079] The control device 400 described above can be realized by incorporating a predetermined program into an information processing device such as the control device 400. Specifically, a program as another aspect of the present invention is a program for causing an information processing device such as the control device 400 to realize processing of detecting, in accordance with image data including the read target as a subject, a predetermined landmark from the image data that can identify a control target to which an imaging device that displays the read target and acquires image data is directed, identifying the control target from the image data according to the detected landmark, and controlling the imaging device so that the imaging direction of the imaging device is oriented toward the identified control target.

[0080] Furthermore, a control method executed by an information processing device such as the control device 400 described above is a method in which the information processing device detects, in accordance with image data including the reading target as a subject, a predetermined landmark from the image data that can identify a control target, which is an object to which an imaging device that displays the reading target and acquires image data is directed, identifies a control target from the image data in accordance with the detected landmark, and controls the imaging device so that the imaging direction of the imaging device is directed toward the identified control target.

[0081] Even if the invention is a program having the above-mentioned configuration, or a recording medium readable by a computer having the program recorded thereon, or a control method, it can achieve the same functions and effects as the above-mentioned control device 400, and therefore can achieve the above-mentioned objective of the present disclosure.

[0082] <Supplementary Notes> Part or all of the above-described embodiments can be described as follows: Below, an outline of the control device and the like in the present invention will be described. However, the present invention is not limited to the following configuration.

[0083] (Supplementary Note 1) A control device comprising: a detection unit that detects, in accordance with image data including a read target as a subject, from the image data a predetermined mark capable of identifying a control target to which an imaging device that displays the read target and acquires the image data is directed, an identification unit that identifies the control target from the image data in accordance with the mark detected by the detection unit, and a control unit that controls the imaging device so that the imaging direction of the imaging device is oriented toward the control target identified by the identification unit. (Supplementary Note 2) The control device according to Supplementary Note 1, wherein the detection unit detects, as the mark, at least one of a predetermined pattern or color that exists within a rectangular object detected from the image data. (Supplementary Note 3) The control device according to Supplementary Note 1 or Supplementary Note 2, wherein the detection unit detects, in the image data, a predetermined object that exists outside the rectangular object detected from the image data. (Supplementary Note 4) The control device according to any one of Supplementary Notes 1 to 3, comprising: a skeleton detection unit that detects a skeleton of a person included in the image data according to the image data; and a candidate identification unit that identifies the control target candidate from the image data according to a detection result by the skeleton detection unit, wherein the identification unit identifies the control target from the candidates identified by the candidate identification unit according to a landmark detected by the detection unit. (Supplementary Note 5) The control device according to Supplementary Note 4, wherein the candidate identification unit identifies the control target candidate according to a posture of the person that can be determined according to a detection result by the skeleton detection unit. (Supplementary Note 6) The control device according to Supplementary Note 4 or Supplementary Note 5, wherein the candidate identification unit identifies the control target candidate according to a distance between a rectangular object detected according to the image data and the skeleton of the person detected by the skeleton detection unit.(Supplementary Note 7) The control device according to any one of Supplementary Notes 1 to 6, wherein the control unit is configured to control a zoom function of the imaging device after the imaging direction of the imaging device is directed toward the control target so that the size of the control target becomes a predetermined size in image data acquired by the imaging device, and the control device comprises: a focus adjustment unit that performs focus adjustment using the image data acquired by the imaging device after zooming, and a reading unit that performs processing to read the read target using the image data acquired by the imaging device after adjustment by the focus adjustment unit. (Supplementary Note 8) The control device according to Supplementary Note 7, wherein the focus adjustment unit performs focus adjustment by adjusting a focus value of the imaging device so that a width of a convex that can be specified in accordance with a change in pixel value at a predetermined position in the image data becomes equal to or less than a predetermined value. (Supplementary Note 9) A control method in which an information processing device, in accordance with image data including a read target as a subject, detects from the image data a predetermined mark capable of identifying a control target to which an imaging device that displays the read target and acquires the image data is directed, identifies the control target from the image data in accordance with the detected mark, and controls the imaging device so that the imaging direction of the imaging device faces the specified control target. (Supplementary Note 10) A program for realizing a process in an information processing device, in accordance with image data including a read target as a subject, detects from the image data a predetermined mark capable of identifying a control target to which an imaging device that displays the read target and acquires the image data is directed, identifies the control target from the image data in accordance with the detected mark, and controls the imaging device so that the imaging direction of the imaging device faces the specified control target.

[0084] Note that some or all of the configurations described in Supplementary Notes 2 to 8 that are dependent on the control device described in Supplementary Note 1 may also be dependent in a similar dependent relationship on the control method described in Supplementary Note 9 and the program described in Supplementary Note 10. Furthermore, not limited to Supplementary Notes 9 and 10, some or all of the configurations described as Supplements may also be dependent on various hardware, software, various recording means for recording software, or systems within the scope of the above-described embodiments.

[0085] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0086] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0087] The present invention claims the benefit of priority based on patent application No. 2024-035762 filed in Japan on March 8, 2024, and all contents described in the patent application are incorporated herein by reference.

[0088] 100 Control system 200 Imaging device 300 Management device 310 Operation input unit 320 Screen display unit 330 Communication interface unit 340 Storage unit 341 Image information 342 Program 350 Arithmetic processing unit 351 Image data acquisition unit 352 Skeleton detection unit 353 Candidate identification unit 354 Marker detection unit 355 Control target identification unit 356 Imaging device control unit 357 Focus adjustment unit 358 Reading unit 400 Control device 401 CPU 402 ROM 403 RAM 404 Program group 405 Storage device 406 Drive device 407 Communication interface 408 Input / output interface 409 Bus 410 Recording medium 411 Communication network 421 Detection unit 422 Identification unit 423 Control unit

Claims

1. A control device having: a detection unit that, in accordance with image data including a read target as a subject, detects from the image data a predetermined mark that can identify a control target to which an imaging device that displays the read target and acquires the image data is directed; an identification unit that identifies the control target from the image data in accordance with the mark detected by the detection unit; and a control unit that controls the imaging device so that the imaging direction of the imaging device faces the direction of the control target identified by the identification unit.

2. The control device according to claim 1, wherein the detection unit detects, as the landmark, at least one of a predetermined pattern or color present in a rectangular object detected from the image data.

3. The control device according to claim 1, wherein the detection unit detects, as the landmark, a predetermined object that exists outside the rectangular object detected from the image data.

4. A control device as described in claim 1, comprising: a skeleton detection unit that detects the skeleton of a person included in the image data according to the image data; and a candidate identification unit that identifies candidates for the control target from the image data according to the detection result by the skeleton detection unit, wherein the identification unit identifies the control target from the candidates identified by the candidate identification unit according to the landmarks detected by the detection unit.

5. The control device according to claim 4, wherein the candidate specification unit specifies the candidate for the control target according to a posture of a person that can be determined according to the detection result by the skeleton detection unit.

6. The control device according to claim 4, wherein the candidate identification unit identifies the candidate for control based on the distance between a rectangular object detected based on the image data and the skeleton of a person detected by the skeleton detection unit.

7. The control device according to claim 1, wherein the control unit is configured to control the zoom function of the imaging device so that the size of the control object becomes a predetermined size in the image data acquired by the imaging device after the imaging direction of the imaging device is turned toward the control object, and the control device has: a focus adjustment unit that adjusts the focus using the image data acquired by the imaging device after zooming; and a reading unit that performs processing to read the read object using the image data acquired by the imaging device after adjustment by the focus adjustment unit.

8. The control device according to claim 7, wherein the focus adjustment unit adjusts the focus by adjusting the focus value of the imaging device so that the width of a convexity that can be identified according to a change in pixel value at a predetermined position in the image data is equal to or less than a predetermined value.

9. A control method in which an information processing device displays a target to be read in accordance with image data including the target as a subject, detects a predetermined landmark from the image data that can identify a control target to which an imaging device that acquires the image data is directed, identifies the control target from the image data in accordance with the detected landmark, and controls the imaging device so that the imaging direction of the imaging device is directed in the direction of the identified control target.

10. A computer-readable recording medium having recorded thereon a program for implementing the following process in an information processing device: displaying the target to be read in accordance with image data including the target as a subject; detecting from the image data a predetermined mark that can identify a control target to which an imaging device that acquires the image data is directed; identifying the control target from the image data in accordance with the detected mark; and controlling the imaging device so that the imaging direction of the imaging device faces the identified control target.

Images